Image processing and printing apparatus

ABSTRACT

An image processing and printing apparatus includes a processor that is configured to process image data to apply effects to the image data. Image data input circuitry receives image data for processing. A card reader reads a card having a two-dimensional code printed on one side of the card. The reader generates data representing executable code upon reading the card. The reader is connected to the processor to write the data to the processor, which executes the code and performs an image processing operation on the image data according to the code. A printhead interface is connected to the processor to receive processed image data from the processor and to generate print data. A printhead is connected to the printhead interface to print processed images.

CROSS-REFERENCES TO RELATED APPLICATIONS

This is a Continuation Application of U.S. application Ser. No.10/269,998, filed on Oct. 15, 2002, now issued U.S. Pat. No. 6,827,282,which is a Continuation Application of U.S. application Ser. No.09/112,781, filed on Jul. 10, 1998, now issued U.S. Pat. No. 6,786,420.

U.S. PAT./PATENT APPLICATION (CLAIMING RIGHT OF CROSS-REFERENCEDPRIORITY FROM AUSTRALIAN AUSTRALIAN PROVISIONAL PATENT PROVISIONALAPPLICATION NO. APPLICATION) DOCKET NO. PO7991 09/113,060 ART01 PO850509/113,070 ART02 PO7988 09/113,073 ART03 PO9395 6,322,181 ART04 PO801709/112,747 ART06 PO8014 09/112,776 ART07 PO8025 09/112,750 ART08 PO803209/112,746 ART09 PO7999 09/112,743 ART10 PO7998 09/112,742 ART11 PO803109/112,741 ART12 PO8030 6,196,541 ART13 PO7997 6,195,150 ART15 PO797909/113,053 ART16 PO8015 09/112,738 ART17 PO7978 09/113,067 ART18 PO798209/113,063 ART19 PO7989 09/113,069 ART20 PO8019 09/112,744 ART21 PO79806,356,715 ART22 PO8018 09/112,777 ART24 PO7938 09/113,224 ART25 PO80166,366,693 ART26 PO8024 09/112,805 ART27 PO7940 09/113,072 ART28 PO793909/112,785 ART29 PO8501 6,137,500 ART30 PO8500 09/112,796 ART31 PO798709/113,071 ART32 PO8022 09/112,824 ART33 PO8497 09/113,090 ART34 PO802009/112,823 ART38 PO8023 09/113,222 ART39 PO8504 09/112,786 ART42 PO800009/113,051 ART43 PO7977 09/112,782 ART44 PO7934 09/113,056 ART45 PO799009/113,059 ART46 PO8499 09/113,091 ART47 PO8502 09/112,753 ART48 PO79816,317,192 ART50 PO7986 09/113,057 ART51 PO7983 09/113,054 ART52 PO802609/112,752 ART53 PO8027 09/112,759 ART54 PO8028 09/112,757 ART56 PO939409/112,758 ART57 PO9396 09/113,107 ART58 PO9397 6,271,931 ART59 PO93986,353,772 ART60 PO9399 6,106,147 ART61 PO9400 09/112,790 ART62 PO94016,304,291 ART63 PO9402 09/112,788 ART64 PO9403 6,305,770 ART65 PO94056,289,262 ART66 PP0959 6,315,200 ART68 PP1397 6,217,165 ART69 PP237009/112,781 DOT01 PP2371 09/113,052 DOT02 PO8003 09/112,834 Fluid01PO8005 09/113,103 Fluid02 PO9404 09/113,101 Fluid03 PO8066 6,227,652IJ01 PO8072 6,213,588 IJ02 PO8040 6,213,589 IJ03 PO8071 6,231,163 IJ04PO8047 6,247,795 IJ05 PO8035 09/113,099 IJ06 PO8044 6,244,691 IJ07PO8063 6,257,704 IJ08 PO8057 09/112,778 IJ09 PO8056 6,220,694 IJ10PO8069 6,257,705 IJ11 PO8049 6,247,794 IJ12 PO8036 6,234,610 IJ13 PO80486,247,793 IJ14 PO8070 6,264,306 IJ15 PO8067 6,241,342 IJ16 PO80016,247,792 IJ17 PO8038 6,264,307 IJ18 PO8033 6,254,220 IJ19 PO80026,234,611 IJ20 PO8068 09/112,808 IJ21 PO8062 6,283,582 IJ22 PO80346,239,821 IJ23 PO8039 09/113,083 IJ24 PO8041 6,247,796 IJ25 PO800409/113,122 IJ26 PO8037 09/112,793 IJ27 PO8043 09/112,794 IJ28 PO804209/113,128 IJ29 PO8064 09/113,127 IJ30 PO9389 6,227,653 IJ31 PO93916,234,609 IJ32 PP0888 6,238,040 IJ33 PP0891 6,188,415 IJ34 PP08906,227,654 IJ35 PP0873 6,209,989 IJ36 PP0993 6,247,791 IJ37 PP089009/112,764 IJ38 PP1398 6,217,153 IJ39 PP2592 09/112,767 IJ40 PP25936,243,113 IJ41 PP3991 6,283,581 IJ42 PP3987 6,247,790 IJ43 PP39856,260,953 IJ44 PP3983 6,267,469 IJ45 PO7935 6,224,780 IJM01 PO79366,235,212 IJM02 PO7937 6,280,643 IJM03 PO8061 6,284,147 IJM04 PO80546,214,244 IJM05 PO8065 6,071,750 IJM06 PO8055 6,267,905 IJM07 PO80536,251,298 IJM08 PO8078 6,258,285 IJM09 PO7933 6,225,138 IJM10 PO79506,241,904 IJM11 PO7949 09/113,129 IJM12 PO8060 09/113,124 IJM13 PO80596,231,773 IJM14 PO8073 6,190,931 IJM15 PO8076 6,248,249 IJM16 PO807509/113,120 IJM17 PO8079 6,241,906 IJM18 PO8050 09/113,116 IJM19 PO80526,241,905 IJM20 PO7948 09/113,117 IJM21 PO7951 6,231,772 IJM22 PO80746,274,056 IJM23 PO7941 09/113,110 IJM24 PO8077 6,248,248 IJM25 PO805809/113,087 IJM26 PO8051 09/113,074 IJM27 PO8045 6,110,754 IJM28 PO795209/113,088 IJM29 PO8046 09/112,771 IJM30 PO9390 6,264,849 IJM31 PO93926,254,793 IJM32 PP0889 6,235,211 IJM35 PP0887 09/112,801 IJM36 PP08826,264,850 IJM37 PP0874 6,258,284 IJM38 PP1396 09/113,098 IJM39 PP39896,228,668 IJM40 PP2591 6,180,427 IJM41 PP3990 6,171,875 IJM42 PP39866,267,904 IJM43 PP3984 6,245,247 IJM44 PP3982 09/112,835 IJM45 PP08956,231,148 IR01 PP0870 09/113,106 IR02 PP0869 09/113,105 IR04 PP088709/113,104 IR05 PP0885 6,238,033 IR06 PP0884 09/112,766 IR10 PP08866,238,111 IR12 PP0871 09/113,086 IR13 PP0876 09/113,094 IR14 PP087709/112,760 IR16 PP0878 6,196,739 IR17 PP0879 09/112,774 IR18 PP08836,270,182 IR19 PP0880 6,152,619 IR20 PP0881 09/113,092 IR21 PO80066,087,638 MEMS02 PO8007 09/113,093 MEMS03 PO8008 09/113,062 MEMS04PO8010 6,041,600 MEMS05 PO8011 09/113,082 MEMS06 PO7947 6,067,797 MEMS07PO7944 09/113,080 MEMS09 PO7946 6,044,646 MEMS10 PO9393 09/113,065MEMS11 PP0875 09/113,078 MEMS12 PP0894 09/113,075 MEMS13

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not applicable.

FIELD OF THE INVENTION

The present invention relates to a data distribution system and inparticular discloses a data distribution mechanism in the form ofDotcards.

BACKGROUND OF THE INVENTION

Methods for distribution of data for automatic reading by computersystems are well known. For example, barcodes are often utilised inconjunction with an optical scanner for the distribution ofcorresponding barcode data. Further, magnetic ink scanning systems haveparticular application on bank cheques which are automatically scannedand the original data determined from the cheque.

There is a general need for a print media scanning system that allowsfor high volumes of computer data to be stored on simple print media,such as a card, and to simultaneously be able to tolerate a high degreeof corruption of the data. For example, the form of distribution cansuffer a number of data corruption errors when the surface is scanned bya scanning device. The errors can include:

1. Dead pixel errors which are a result of reading the surface of thecard with a linear CCD having a faulty pixel reader for a line therebyproducing the same value for all points on the line.

2. The system adopted should tolerate writing errors wherein text iswritten by the owner of the card on the surface. Such text writingerrors are ideally tolerated by any scanning system scanning the card.

3. Various data errors on the surface of the card may rise and anyscuffs or blotches should be tolerated by any system determining theinformation stored on the surface of the card.

4. A certain degree of “play” exists in the insertion of the card into acard reader. This play can comprise a degree of rotation of the cardwhen read by a card reader.

5. Further, the card reader is assumed to be driven past a CCD typescanner device by means of an electric motor. The electric motor mayexperience a degree of fluctuation which will result in fluctuations inthe rate of transmission of the data across the surface of the CCD.These motor fluctuation errors should also be tolerated by the dataencoding method on the surface of the card.

6. The scanner of the surface of the card may experience various devicefluctuations such that the intensity of individual pixels may vary.Reader intensity variations should also be accounted for in any systemor method implemented in the data contained on the surface of the card.

Many forms of condensed information storage are well known. For example,in the field of computer devices, it is common to utilize magnetic discdrives which can be of a fixed or portable nature. In respect ofportable discs, “Floppy Discs”, “Zip Discs”, and other forms of portablemagnetic storage media have achieved a large degree of acceptance on themarket place.

Another form of portable storage is the compact disc “CD” which utilizesa series of elongated pits along a spiral track which is read by a laserbeam device. The utilization of Compact Disks provides for an extremelylow cost form of storage. However, the technologies involved are quitecomplex and the use of rewritable CD type devices is extremely limited.

Other forms of storage include magnetic cards, often utilized for creditcards or the like. These cards normally have a magnetic strip on theback for recording information which is of relevance to the card user.Recently, the convenience of magnetic cards has been extended in theform of SmartCard technology which includes incorporation of integratedcircuit type devices on to the card. Unfortunately, the cost of suchdevices is often high and the complexity of the technology utilized canalso be significant.

SUMMARY OF THE INVENTION

According to a first aspect of the invention, there is provided an imageprocessing and printing apparatus which includes

a processor that is configured to process image data to apply effects tothe image data;

image data input circuitry for receiving image data for processing;

a card reader for reading a card having a two-dimensional code printedon one side of the card, configured to generate data representingexecutable code upon reading the card, and connected to the processor towrite the data to the processor, which is configured to execute the codeand to perform an image processing operation on the image data accordingto the code;

a printhead interface connected to the processor to receive processedimage data from the processor and to generate print data; and

a printhead connected to the printhead interface to print processedimages.

The image data input circuitry may include a memory which is connectedto an image capture device to receive image data from the image capturedevice, the memory being accessible by the processor to write the imagedata to the processor. The memory may be a random access memory.

The card reader may include a linear sensor having a scanning resolutionwhich is at least twice a resolution of the two-dimensional code on thecard.

The apparatus may include a reader motor for propelling the card pastthe linear sensor at a substantially constant rate.

The processor may be configured to detect, via the card reader, a dataarea on the card and subsequently to detect a bit pattern represented bythe two-dimensional code in the data area. The processor may further beconfigured to perform a decoding operation on the bit pattern togenerate redundancy encoded data and then to decode the redundancyencoded data to generate the executable code.

The printhead may be a pagewidth inkjet printhead.

In accordance with a second aspect of the present invention, there isprovided an identifying card comprising: a first surface carrying humanreadable information relevant to an owner of the identifying card; and asecond, opposed surface carrying encoded information encoded in a highlyfault tolerant manner, said encoded information being adapted forsensing by a sensing device and decoded by a computational processor, soas to provide information relevant to the owner in a human readableform, the encoded information comprising an array of dots applied tosaid second surface; wherein the encoded information comprises spatiallydistributed redundancy encoded data such that the information is encodedin a highly fault tolerant manner and can be decoded by said processordespite a localized obliteration of the encoded information on the card.

BRIEF DESCRIPTION OF THE DRAWINGS

Notwithstanding any other forms which may fall within the scope of thepresent invention, preferred forms of the invention will now bedescribed, by way of example only, with reference to the accompanyingdrawings in which:

FIG. 1 illustrates an Artcam device constructed in accordance with thepreferred embodiment;

FIG. 2 is a schematic block diagram of the main Artcam electroniccomponents;

FIG. 3 illustrates a time line of the process of sampling an Artcard;

FIG. 4 illustrates the super sampling process;

FIG. 5 illustrates the process of reading a rotated Artcard,

FIG. 6 illustrates a flow chart of the steps necessary to decode anArtcard,

FIG. 7 illustrates an enlargement of the left hand corner of a singleArtcard,

FIG. 8 illustrates a single target for detection;

FIG. 9 illustrates the method utilised to detect targets;

FIG. 10 illustrates the method of calculating the distance between twotargets;

FIG. 11 illustrates the process of centroid drift;

FIG. 12 shows one form of centroid lookup table;

FIG. 13 illustrates the centroid updating process;

FIG. 14 illustrates a delta processing lookup table utilised in thepreferred embodiment;

FIG. 15 illustrates the process of unscrambling Artcard data;

FIG. 16 illustrates a magnified view of a series of dots;

FIG. 17 illustrates the data surface of a dot card;

FIG. 18 illustrates schematically the layout of a single datablock;

FIG. 19 illustrates a single datablock;

FIG. 20 and FIG. 21 illustrate magnified views of portions of thedatablock of FIG. 19;

FIG. 22 illustrates a single target structure;

FIG. 23 illustrates the target structure of a datablock;

FIG. 24 illustrates the positional relationship of targets relative toborder clocking regions of a data region;

FIG. 25 illustrates the orientation columns of a datablock;

FIG. 26 illustrates the array of dots of a datablock;

FIG. 27 illustrates schematically the structure of data for Reed-Solomonencoding;

FIG. 28 illustrates an example Reed-Solomon encoding;

FIG. 29 illustrates the Reed-Solomon encoding process;

FIG. 30 illustrates the layout of encoded data within a datablock;

FIG. 31 illustrates the sampling process in sampling an alternativeArtcard;

FIG. 32 illustrates, in exaggerated form, an example of sampling arotated alternative Artcard;

FIG. 33 illustrates the scanning process;

FIG. 34 illustrates the likely scanning distribution of the scanningprocess;

FIG. 35 illustrates the relationship between probability of symbolerrors and Reed-Solomon block errors;

FIG. 36 illustrates a flow chart of the decoding process;

FIG. 37 illustrates a process utilization diagram of the decodingprocess;

FIG. 38 illustrates the dataflow steps in decoding;

FIG. 39 illustrates the reading process in more detail;

FIG. 40 illustrates the process of detection of the start of analternative Artcard in more detail;

FIG. 41 illustrates the extraction of bit data process in more detail;

FIG. 42 illustrates the segmentation process utilized in the decodingprocess;

FIG. 43 illustrates the decoding process of finding targets in moredetail;

FIG. 44 illustrates the data structures utilized in locating targets;

FIG. 45 illustrates the Lancos 3 function structure;

FIG. 46 illustrates an enlarged portion of a datablock illustrating theclockmark and border region;

FIG. 47 illustrates the processing steps in decoding a bit image;

FIG. 48 illustrates the dataflow steps in decoding a bit image;

FIG. 49 illustrates the descrambling process of the preferredembodiment;

FIG. 50 illustrates the process of generating an 8 bit dot output;

FIG. 51 illustrates a perspective view of the card reader;

FIG. 52 illustrates an exploded perspective of a card reader;

FIG. 53 illustrates a close up view of the Artcard reader,

FIG. 54 illustrates a perspective view of the print roll and print head;

FIG. 55 illustrates a first exploded perspective view of the print roll;

FIG. 56 illustrates a second exploded perspective view of the printroll;

FIG. 57 illustrates the print roll authentication chip

FIG. 58 illustrates an enlarged view of the print roll authenticationchip;

DESCRIPTION OF PREFERRED AND OTHER EMBODIMENTS

The digital image processing camera system constructed in accordancewith the preferred embodiment is as illustrated in FIG. 1. The cameraunit 1 includes means for the insertion of an integral print roll (notshown). The camera unit 1 can include an area image sensor 2 whichsensors an image 3 for captured by the camera. Optionally, the secondarea image sensor can be provided to also image the scene 3 and tooptionally provide for the production of stereographic output effects.

The camera 1 can include an optional color display 5 for the display ofthe image being sensed by the sensor 2. When a simple image is beingdisplayed on the display 5, the button 6 can be depressed resulting inthe printed image 8 being output by the camera unit 1. A series ofcards, herein after known as “Artcards” 9 contain, on one surfaceencoded information and on the other surface, contain an image distortedby the particular effect produced by the Artcard 9. The Artcard 9 isinserted in an Artcard reader 10 in the side of camera 1 and, uponinsertion, results in output image 8 being distorted in the same manneras the distortion appearing on the surface of Artcard 9. Hence, by meansof this simple user interface a user wishing to produce a particulareffect can insert one of many Artcards 9 into the Artcard reader 10 andutilize button 19 to take a picture of the image 3 resulting in acorresponding distorted output image 8.

The camera unit 1 can also include a number of other control button 13,14 in addition to a simple LCD output display 15 for the display ofinformative information including the number of printouts left on theinternal print roll on the camera unit. Additionally, different outputformats can be controlled by CHP switch 17.

Turning now to FIG. 2, there is illustrated a schematic view of theinternal hardware of the camera unit 1. The internal hardware is basedaround an Artcam central processor unit (ACP) 31.

Artcam Central Processor 31

The Artcam central processor 31 provides many functions which form the‘heart’ of the system. The ACP 31 is preferably implemented as acomplex, high speed, CMOS system on-a-chip. Utilising standard celldesign with some full custom regions is recommended. Fabrication on a0.25μ CMOS process will provide the density and speed required, alongwith a reasonably small die area.

The functions provided by the ACP 31 include:

1. Control and digitization of the area image sensor 2. A 3Dstereoscopic version of the ACP requires two area image sensorinterfaces with a second optional image sensor 4 being provided forstereoscopic effects.

2. Area image sensor compensation, reformatting, and image enhancement.

3. Memory interface and management to a memory store 33.

4. Interface, control, and analog to digital conversion of an Artcardreader linear image sensor 34 which is provided for the reading of datafrom the Artcards 9.

5. Extraction of the raw Artcard data from the digitized and encodedArtcard image.

6. Reed-Solomon error detection and correction of the Artcard encodeddata. The encoded surface of the Artcard 9 includes information on howto process an image to produce the effects displayed on the imagedistorted surface of the Artcard 9. This information is in the form of ascript, hereinafter known as a “Vark script”. The Vark script isutilised by an interpreter running within the ACP 31 to produce thedesired effect

7. Interpretation of the Vark script on the Artcard 9.

8. Performing image processing operations as specified by the Varkscript.

9. Controlling various motors for the paper transport 36, zoom lens 38,autofocus 39 and Artcard driver 37.

10. Controlling a guillotine actuator 40 for the operation of aguillotine 41 for the cutting of photographs 8 from print roll 42.

11. Half-toning of the image data for printing.

12. Providing the print data to a print-head 44 at the appropriatetimes.

13. Controlling the print head 44.

14. Controlling the ink pressure feed to print-head 44.

15. Controlling optional flash unit 56.

16. Reading and acting on various sensors in the camera, includingcamera orientation sensor 46, autofocus 47 and Artcard insertion sensor49.

17. Reading and acting on the user interface buttons 6, 13, 14.

18. Controlling the status display 15.

19. Providing viewfinder and preview images to the color display 5.

20. Control of the system power consumption, including the ACP powerconsumption via power management circuit 51.

21. Providing external communications 52 to general purpose computers(using part USB).

22. Reading and storing information in a printing roll authenticationchip 53.

23. Reading and storing information in a camera authentication chip 54.

24. Communicating with an optional mini-keyboard 57 for textmodification.

Quartz Crystal 58

A quartz crystal 58 is used as a frequency reference for the systemclock. As the system clock is very high, the ACP 31 includes a phaselocked loop clock circuit to increase the frequency derived from thecrystal 58.

Artcard 9

The Artcard 9 is a program storage medium for the Artcam unit. As notedpreviously, the programs are in the form of Vark scripts. Vark is apowerful image processing language especially developed for the Artcamunit. Each Artcard 9 contains one Vark script, and thereby defines oneimage processing style.

Preferably, the VARK language is highly image processing specific. Bybeing highly image processing specific, the amount of storage requiredto store the details on the card are substantially reduced. Further, theease with which new programs can be created, including enhanced effects,is also substantially increased. Preferably, the language includesfacilities for handling many image processing functions including imagewarping via a warp map, convolution, color lookup tables, posterizing animage, adding noise to an image, image enhancement filters, paintingalgorithms, brush jittering and manipulation edge detection filters,tiling, illumination via light sources, bump maps, text, face detectionand object detection attributes, fonts, including three dimensionalfonts, and arbitrary complexity pre-rendered icons. Further details ofthe operation of the Vark language interpreter are containedhereinafter.

Hence, by utilizing the language constructs as defined by the createdlanguage, new affects on arbitrary images can be created and constructedfor inexpensive storage on Artcard and subsequent distribution to cameraowners. Further, on one surface of the card can be provided an exampleillustrating the effect that a particular VARK script, stored on theother surface of the card, will have on an arbitrary captured image.

By utilizing such a system, camera technology can be distributed withouta great fear of obsolescence in that, provided a VARK interpreter isincorporated in the camera device, a device independent scenario isprovided whereby the underlying technology can be completely varied overtime. Further, the VARK scripts can be updated as new filters arecreated and distributed in an inexpensive manner, such as via simplecards for card reading.

The Artcard 9 is a piece of thin white plastic with the same format as acredit card (86 mm long by 54 mm wide). The Artcard is printed on bothsides using a high resolution ink jet printer. The inkjet printertechnology is assumed to be the same as that used in the Artcam, with1600 dpi (63 dpmm) resolution. A major feature of the Artcard 9 is lowmanufacturing cost. Artcards can be manufactured at high speeds as awide web of plastic film. The plastic web is coated on both sides with ahydrophilic dye fixing layer. The web is printed simultaneously on bothsides using a ‘pagewidth’ color ink jet printer. The web is then cut andpunched into individual cards. On one face of the card is printed ahuman readable representation of the effect the Artcard 9 will have onthe sensed image. This can be simply a standard image which has beenprocessed using the Vark script stored on the back face of the card.

On the back face of the card is printed an array of dots which can bedecoded into the Vark script that defines the image processing sequence.The print area is 80 mm×50 mm, giving a total of 15,876,000 dots. Thisarray of dots could represent at least 1.89 Mbytes of data. To achievehigh reliability, extensive error detection and correction isincorporated in the array of dots. This allows a substantial portion ofthe card to be defaced, worn, creased, or dirty with no effect on dataintegrity. The data coding used is Reed-Solomon coding, with half of thedata devoted to error correction. This allows the storage of 967 Kbytesof error corrected data on each Artcard 9.

Linear Image Sensor 34

The Artcard linear sensor 34 converts the aforementioned Artcard dataimage to electrical signals. As with the area image sensor 2, 4, thelinear image sensor can be fabricated using either CCD or APS CMOStechnology. The active length of the image sensor 34 is 50 mm, equal tothe width of the data array on the Artcard 9. To satisfy Nyquist'ssampling theorem, the resolution of the linear image sensor 34 must beat least twice the highest spatial frequency of the Artcard opticalimage reaching the image sensor. In practice, data detection is easierif the image sensor resolution is substantially above this. A resolutionof 4800 dpi (189 dpmm) is chosen, giving a total of 9,450 pixels. Thisresolution requires a pixel sensor pitch of 5.3 μm. This can readily beachieved by using four staggered rows of 20 μm pixel sensors.

The linear image sensor is mounted in a special package which includes aLED 65 to illuminate the Artcard 9 via a light-pipe (not shown).

The Artcard reader light-pipe can be a molded light-pipe which hasseveral function:

1. It diffuses the light from the LED over the width of the card usingtotal internal reflection facets.

2. It focuses the light onto a 16 μm wide strip of the Artcard 9 usingan integrated cylindrical lens.

3. It focuses light reflected from the Artcard onto the linear imagesensor pixels using a molded array of microlenses.

The operation of the Artcard reader is explained further hereinafter.

Artcard Reader Motor 37

The Artcard reader motor propels the Artcard past the linear imagesensor 34 at a relatively constant rate. As it may not be cost effectiveto include extreme precision mechanical components in the Artcardreader, the motor 37 is a standard miniature motor geared down to anappropriate speed to drive a pair of rollers which move the Artcard 9.The speed variations, rumble, and other vibrations will affect the rawimage data as circuitry within the APC 31 includes extensivecompensation for these effects to reliably read the Artcard data.

The motor 37 is driven in reverse when the Artcard is to be ejected.

Artcard Motor Driver 61

The Artcard motor driver 61 is a small circuit which amplifies thedigital motor control signals from the APC 31 to levels suitable fordriving the motor 37.

Card Insertion Sensor 49

The card insertion sensor 49 is an optical sensor which detects thepresence of a card as it is being inserted in the card reader 34. Upon asignal from this sensor 49, the APC 31 initiates the card readingprocess, including the activation of the Artcard reader motor 37.

Card Eject Button 16

A card eject button 16 (FIG. 1) is used by the user to eject the currentArtcard, so that another Artcard can be inserted. The APC 31 detects thepressing of the button, and reverses the Artcard reader motor 37 toeject the card.

Card Status Indicator 66

A card status indicator 66 is provided to signal the user as to thestatus of the Artcard reading process. This can be a standard bi-color(red/green) LED. When the card is successfully read, and data integrityhas been verified, the LED Lights up green continually. If the card isfaulty, then the LED lights up red.

If the camera is powered from a 1.5 V instead of 3V battery, then thepower supply voltage is less than the forward voltage drop of the greedLED, and the LED will not light. In this case, red LEDs can be used, orthe LED can be powered from a voltage pump which also powers othercircuits in the Artcam which require higher voltage.

64 Mbit DRAM 33

To perform the wide variety of image processing effects, the camerautilizes 8 Mbytes of memory 33. This can be provided by a single 64 Mbitmemory chip. Of course, with changing memory technology increased Dramstorage sizes may be substituted.

High speed access to the memory chip is required. This can be achievedby using a Rambus DRAM (burst access rate of 500 Mbytes per second) orchips using the new open standards such as double data rate (DDR) SDRAMor Synclink DRAM.

Inserting an Artcard

When a user inserts an Artcard 9, the Artcard Sensor 49 detects itnotifying the ACP72. This results in the software inserting an ‘ArtcardInserted’ event into the event queue. When the event is processedseveral things occur:

The current Artcard is marked as invalid (as opposed to ‘none’).

The Print Image is marked as invalid.

The Artcard motor 37 is started up to load the Artcard

The Artcard Interface 87 is instructed to read the Artcard

The Artcard Interface 87 accepts signals from the Artcard scanner linearCCD 34, detects the bit pattern printed on the card, and corrects errorsin the detected bit pattern, producing a valid Artcard data block inDRAM.

Reading Data From the Artcard CCD—General Considerations

As illustrated in FIG. 3, the Data Card reading process has 4 phasesoperated while the pixel data is read from the card. The phases are asfollows:

Phase 1. Detect data area on Artcard

Phase 2. Detect bit pattern from Artcard based on CCD pixels, and writeas bytes.

Phase 3. Descramble and XOR the byte-pattern

Phase 4. Decode data (Reed-Solomon decode)

As illustrated in FIG. 4, the Artcard 9 must be sampled at least atdouble the printed resolution to satisfy Nyquist's Theorem. In practiceit is better to sample at a higher rate than this. Preferably, thepixels are sampled 230 at 3 times the resolution of a printed dot ineach dimension, requiring 9 pixels to define a single dot. Thus if theresolution of the Artcard 9 is 1600 dpi, and the resolution of thesensor 34 is 4800 dpi, then using a 50 mm CCD image sensor results in9450 pixels per column. Therefore if we require 2 MB of dot data (at 9pixels per dot) then this requires 2 MB*8*9/9450=15,978columns=approximately 16,000 columns. Of course if a dot is not exactlyaligned with the sampling CCD the worst and most likely case is that adot will be sensed over a 16 pixel area (4×4) 231.

An Artcard 9 may be slightly warped due to heat damage, slightly rotated(up to, say 1 degree) due to differences in insertion into an Artcardreader, and can have slight differences in true data rate due tofluctuations in the speed of the reader motor 37. These changes willcause columns of data from the card not to be read as correspondingcolumns of pixel data. As illustrated in FIG. 5, a 1 degree rotation inthe Artcard 9 can cause the pixels from a column on the card to be readas pixels across 166 columns:

Finally, the Artcard 9 should be read in a reasonable amount of timewith respect to the human operator. The data on the Artcard covers mostof the Artcard surface, so timing concerns can be limited to the Artcarddata itself. A reading time of 1.5 seconds is adequate for Artcardreading.

The Artcard should be loaded in 1.5 seconds. Therefore all 16,000columns of pixel data must be read from the CCD 34 in 1.5 second, i.e.10,667 columns per second. Therefore the time available to read onecolumn is 1/10667 seconds, or 93,747 ns. Pixel data can be written tothe DRAM one column at a time, completely independently from anyprocesses that are reading the pixel data.

The time to write one column of data (9450/2 bytes since the reading canbe 4 bits per pixel giving 2×4 bit pixels per byte) to DRAM is reducedby using 8 cache lines. If 4 lines were written out at one time, the 4banks can be written to independently, and thus overlap latency reduced.Thus the 4725 bytes can be written in 11,840 ns (4725/128*320 ns). Thusthe time taken to write a given column's data to DRAM uses just under13% of the available bandwidth.

Decoding an Artcard

A simple look at the data sizes shows the impossibility of fitting theprocess into the 8 MB of memory 33 if the entire Artcard pixel data (140MB if each bit is read as a 3×3 array) as read by the linear CCD 34 iskept. For this reason, the reading of the linear CCD, decoding of thebitmap, and the un-bitmap process should take place in real-time (whilethe Artcard 9 is traveling past the linear CCD 34), and these processesmust effectively work without having entire data stores available.

When an Artcard 9 is inserted, the old stored Print Image and anyexpanded Photo Image becomes invalid. The new Artcard 9 can containdirections for creating a new image based on the currently capturedPhoto Image. The old Print Image is invalid, and the area holdingexpanded Photo Image data and image pyramid is invalid, leaving morethan 5 MB that can be used as scratch memory during the read process.Strictly speaking, the 1 MB area where the Artcard raw data is to bewritten can also be used as scratch data during the Artcard read processas long as by the time the final Reed-Solomon decode is to occur, that 1MB area is free again. The reading process described here does not makeuse of the extra 1 MB area (except as a final destination for the data).

It should also be noted that the unscrambling process requires two setsof 2 MB areas of memory since unscrambling cannot occur in place.Fortunately the 5 MB scratch area contains enough space for thisprocess.

Turning now to FIG. 6, there is shown a flowchart 220 of the stepsnecessary to decode the Artcard data. These steps include reading in theArtcard 221, decoding the read data to produce corresponding encodedXORed scrambled bitmap data 223. Next a checkerboard XOR is applied tothe data to produces encoded scrambled data 224. This data is thenunscrambled 227 to produce data 225 before this data is subjected toReed-Solomon decoding to produce the original raw data 226.Alternatively, unscrambling and XOR process can take place together, notrequiring a separate pass of the data. Each of the above steps isdiscussed in further detail hereinafter. As noted previously withreference to FIG. 6, the Artcard Interface, therefore, has 4 phases, thefirst 2 of which are time-critical, and must take place while pixel datais being read from the CCD:

Phase 1. Detect data area on Artcard

Phase 2. Detect bit pattern from Artcard based on CCD pixels, and writeas bytes.

Phase 3. Descramble and XOR the byte-pattern

Phase 4. Decode data (Reed-Solomon decode)

The four phases are described in more detail as follows:

Phase 1. As the Artcard 9 moves past the CCD 34 the AI must detect thestart of the data area by robustly detecting special targets on theArtcard to the left of the data area. If these cannot be detected, thecard is marked as invalid. The detection must occur in real-time, whilethe Artcard 9 is moving past the CCD 34.

If necessary, rotation invariance can be provided. In this case, thetargets are repeated on the right side of the Artcard, but relative tothe bottom right corner instead of the top corner. In this way thetargets end up in the correct orientation if the card is inserted the“wrong” way. Phase 3 below can be altered to detect the orientation ofthe data, and account for the potential rotation.

Phase 2. Once the data area has been determined, the main read processbegins, placing pixel data from the CCD into an ‘Artcard data window’,detecting bits from this window, assembling the detected bits intobytes, and constructing a byte-image in DRAM. This must all be donewhile the Artcard is moving past the CCD.

Phase 3. Once all the pixels have been read from the Artcard data area,the Artcard motor 37 can be stopped, and the byte image descrambled andXORed. Although not requiring real-time performance, the process shouldbe fast enough not to annoy the human operator. The process must take 2MB of scrambled bit-image and write the unscrambled/XORed bit-image to aseparate 2 MB image.

Phase 4. The final phase in the Artcard read process is the Reed-Solomondecoding process, where the 2 MB bit-image is decoded into a 1 MB validArtcard data area. Again, while not requiring real-time performance itis still necessary to decode quickly with regard to the human operator.If the decode process is valid, the card is marked as valid. If thedecode failed, any duplicates of data in the bit-image are attempted tobe decoded, a process that is repeated until success or until there areno more duplicate images of the data in the bit image.

The four phase process described requires 4.5 MB of DRAM. 2 MB isreserved for Phase 2 output, and 0.5 MB is reserved for scratch dataduring phases 1 and 2. The remaining 2 MB of space can hold over 440columns at 4725 byes per column. In practice, the pixel data being readis a few columns ahead of the phase 1 algorithm, and in the worst case,about 180 columns behind phase 2, comfortably inside the 440 columnlimit A description of the actual operation of each phase will now beprovided in greater detail.

Phase 1—Detect Data Area on Artcard

This phase is concerned with robustly detecting the left-hand side ofthe data area on the Artcard 9. Accurate detection of the data area isachieved by accurate detection of special targets printed on the leftside of the card. These targets are especially designed to be easy todetect even if rotated up to 1 degree.

Turning to FIG. 7, there is shown an enlargement of the left hand sideof an Artcard 9. The side of the card is divided into 16 bands, 239 witha target eg. 241 located at the center of each band. The bands arelogical in that there is no line drawn to separate bands. Turning toFIG. 8, there is shown a single target 241. The target 241, is a printedblack square containing a single white dot. The idea is to detectfirstly as many targets 241 as possible, and then to join at least 8 ofthe detected white-dot locations into a single logical straight line. Ifthis can be done, the start of the data area 243 is a fixed distancefrom this logical line. If it cannot be done, then the card is rejectedas invalid.

As shown in FIG. 7, the height of the card 9 is 3150 dots. A target(Target0) 241 is placed a fixed distance of 24 dots away from the topleft corner 244 of the data area so that it falls well within the firstof 16 equal sized regions 239 of 192 dots (576 pixels) with no target inthe final pixel region of the card. The target 241 must be big enough tobe easy to detect, yet be small enough not to go outside the height ofthe region if the card is rotated 1 degree. A suitable size for thetarget is a 31×31 dot (93×93 sensed pixels) black square 241 with thewhite dot 242.

At the worst rotation of 1 degree, a 1 column shift occurs every 57pixels. Therefore in a 590 pixel sized band, we cannot place any part ofour symbol in the top or bottom 12 pixels or so of the band or theycould be detected in the wrong band at CCD read time if the card isworst case rotated.

Therefore, if the black part of the rectangle is 57 pixels high (19dots) we can be sure that at least 9.5 black pixels will be read in thesame column by the CCD (worst case is half the pixels are in one columnand half in the next). To be sure of reading at least 10 black dots inthe same column, we must have a height of 20 dots. To give room forerroneous detection on the edge of the start of the black dots, weincrease the number of dots to 31, giving us 15 on either side of thewhite dot at the target's local coordinate (15, 15). 31 dots is 91pixels, which at most suffers a 3 pixel shift in column, easily withinthe 576 pixel band.

Thus each target is a block of 31×31 dots (93×93 pixels) each with thecomposition:

15 columns of 31 black dots each (45 pixel width columns of 93 pixels).

1 column of 15 black dots (45 pixels) followed by 1 white dot (3 pixels)and then a further 15 black dots (45 pixels)

15 columns of 31 black dots each (45 pixel width columns of 93 pixels)

Detect Targets

Targets are detected by reading columns of pixels, one column at a timerather than by detecting dots. It is necessary to look within a givenband for a number of columns consisting of large numbers of contiguousblack pixels to build up the left side of a target. Next, it is expectedto see a white region in the center of further black columns, andfinally the black columns to the left of the target center.

Eight cache lines are required for good cache performance on the readingof the pixels. Each logical read fills 4 cache lines via 4 sub-readswhile the other 4 cache-lines are being used. This effectively uses up13% of the available DRAM bandwidth.

As illustrated in FIG. 9, the detection mechanism FIFO for detecting thetargets uses a filter 245, run-length encoder 246, and a FIFO 247 thatrequires special wiring of the top 3 elements (S1, S2, and S3) forrandom access.

The columns of input pixels are processed one at a time until either allthe targets are found, or until a specified number of columns have beenprocessed. To process a column, the pixels are read from DRAM, passedthrough a filter 245 to detect a 0 or 1, and then run length encoded246. The bit value and the number of contiguous bits of the same valueare placed in FIFO 247. Each entry of the FIFO 249 is in 8 bits, 7 bits250 to hold the run-length, and 1 bit 249 to hold the value of the bitdetected.

The run-length encoder 246 only encodes contiguous pixels within a 576pixel (192 dot) region.

The top 3 elements in the FIFO 247 can be accessed 252 in any randomorder. The run lengths (in pixels) of these entries are filtered into 3values: short, medium, and long in accordance with the following table:

Short Used to detect white dot. RunLength < 16 Medium Used to detectruns of black 16 <= RunLength < 48 above or below the white dot in thecenter of the target. Long Used to detect run lengths of RunLength >= 48black to the left and right of the center dot in the target.Looking at the top three entries in the FIFO 247 there are 3 specificcases of interest:

Case 1 S1 = white long We have detected a black column of the S2 = blacklong target to the left of or to the right of the S3 = white whitecenter dot. medium/long Case 2 S1 = white long If we've been processinga series of S2 = black medium columns of Case 1s, then we have S3 =white short probably detected the white dot in this Previous 8 columnscolumn. We know that the next entry will were Case 1 be black (or itwould have been included in the white S3 entry), but the number of blackpixels is in question. Need to verify by checking after the next FIFOadvance (see Case 3). Case 3 Prev = Case 2 We have detected part of thewhite S3 = black med dot. We expect around 3 of these, and then somemore columns of Case 1.Preferably, the following information per region band is kept:

TargetDetected  1 bit BlackDetectCount  4 bits WhiteDetectCount  3 bitsPrevColumnStartPixel 15 bits TargetColumn ordinate 16 bits (15:1)TargetRow ordinate 16 bits (15:1) TOTAL  7 bytes (rounded to 8 bytes foreasy addressing)

Given a total of 7 bytes. It makes address generation easier if thetotal is assumed to be 8 bytes. Thus 16 entries requires 16*8=128 bytes,which fits in 4 cache lines. The address range should be inside thescratch 0.5 MB DRAM area since other phases make use of the remaining 4MB data area.

When beginning to process a given pixel column, the register valueS2Startpixel 254 is reset to 0. As entries in the FIFO advance from S2to S1, they are also added 255 to the existing S2Startpixel value,giving the exact pixel position of the run currently defined in S2.Looking at each of the 3 cases of interest in the FIFO, S2Startpixel canbe used to determine the start of the black area of a target (Cases 1and 2), and also the start of the white dot in the center of the target(Case 3). An algorithm for processing columns can be as follows:

1 TargetDetected[0–15] := 0 BlackDetectCount[0–15] := 0WhiteDetectCount[0–15] := 0 TargetRow[0–15] := 0 TargetColumn[0–15] := 0PrevColStartPixel[0–15] := 0 CurrentColumn := 0 2 Do ProcessColumn 3CurrentColumn++ 4 If (CurrentColumn <= LastValidColumn) Goto 2The steps involved in the processing a column (Process Column) are asfollows:

1 S2StartPixel := 0 FIFO := 0 BlackDetectCount := 0 WhiteDetectCount :=0 ThisColumnDetected := FALSE PrevCaseWasCase2 := FALSE 2 If(!TargetDetected[Target]) & (! ColumnDetected[Target])  ProcessCases EndIf3 PrevCaseWasCase2 := Case=2 4 Advance FIFO

The processing for each of the 3 (Process Cases) cases is as follows:

Case 1: BlackDetectCount[target] < 8 √ := ABS(S2StartPixel − ORPrevColStartPixel[Target]) WhiteDetectCount[Target] = 0 If (0<=√ < 2) BlackDetectCount[Target]++ (max value =8) Else BlackDetectCount[Target] := 1  WhiteDetectCount[Target] := 0 EndIfPrevColStartPixel[Target] := S2StartPixel ColumnDetected[Target] := TRUEBitDetected = 1 BlackDetectCount[target] >= 8 PrevColStartPixel[Target]:= S2StartPixel WhiteDetectCount[Target] != 0 ColumnDetected[Target] :=TRUE BitDetected = 1 TargetDetected[Target] := TRUE TargetColumn[Target]:= CurrentColumn − 8 −     (WhiteDetectCount[Target]/2)Case 2:

No special processing is recorded except for setting the‘PrevCaseWasCase2’ flag for identifying Case 3 (see Step 3 of processinga column described above)

Case 3: PrevCaseWasCase2 = TRUE If (WhiteDetectCount[Target] < 2)BlackDetectCount[Target] >= 8  TargetRow[Target] = S2StartPixel +WhiteDetectCount=1  (S2_(RunLength)/2) EndIf √ := ABS(S2StartPixel −PrevColStartPixel[Target]) If (0<=√ < 2)  WhiteDetectCount[Target]++Else  WhiteDetectCount[Target] := 1 EndIf PrevColStartPixel[Target] :=S2StartPixel ThisColumnDetected := TRUE BitDetected = 0

At the end of processing a given column, a comparison is made of thecurrent column to the maximum number of columns for target detection. Ifthe number of columns allowed has been exceeded, then it is necessary tocheck how many targets have been found. If fewer than 8 have been found,the card is considered invalid.

Process Targets

After the targets have been detected, they should be processed. All thetargets may be available or merely some of them. Some targets may alsohave been erroneously detected.

This phase of processing is to determine a mathematical line that passesthrough the center of as many targets as possible. The more targets thatthe line passes through, the more confident the target position has beenfound. The limit is set to be 8 targets. If a line passes through atleast 8 targets, then it is taken to be the right one.

It is all right to take a brute-force but straightforward approach sincethere is the time to do so (see below), and lowering complexity makestesting easier. It is necessary to determine the line between targets 0and 1 (if both targets are considered valid) and then determine how manytargets fall on this line. Then we determine the line between targets 0and 2, and repeat the process. Eventually we do the same for the linebetween targets 1 and 2, 1 and 3 etc. and finally for the line betweentargets 14 and 15. Assuming all the targets have been found, we need toperform 15+14+13+ . . . =90 sets of calculations (with each set ofcalculations requiring 16 tests=1440 actual calculations), and choosethe line which has the maximum number of targets found along the line.The algorithm for target location can be as follows:

TargetA := 0 MaxFound := 0 BestLine := 0 While (TargetA < 15)   If(TargetA is Valid)   TargetB:= TargetA + 1   While (TargetB<= 15)   If(TargetB is valid)   CurrentLine := line between TargetA and TargetB  TargetC := 0;   While (TargetC <= 15)     If (TargetC valid ANDTargetC on line AB)       TargetsHit++     EndIf     If (TargetsHit >MaxFound)       MaxFound := TargetsHit       BestLine := CurrentLine    EndIf       TargetC++     EndWhile       EndIf   TargetB ++    EndWhile   EndIf   TargetA++ EndWhile If (MaxFound < 8)   Card isInvalid Else   Store expected centroids for rows based on BestLine EndIf

As illustrated in FIG. 3, in the algorithm above, to determine aCurrentLine 260 from Target A 261 and target B, it is necessary tocalculate Δrow (264) & Δcolumn (263) between targets 261, 262, and thelocation of Target A. It is then possible to move from Target 0 toTarget 1 etc. by adding Δrow and Δcolumn. The found (if actually found)location of target N can be compared to the calculated expected positionof Target N on the line, and if it falls within the tolerance, thenTarget N is determined to be on the line.

To calculate Δrow & Δcolumn:Δrow=(row_(TargetA)−row_(TargetB))/(B−A)Δcolumn=(column_(TargetA)−column_(TargetB))/(B−A)Then we calculate the position of Target0:row=rowTargetA−(A*Δrow)column=columnTargetA−(A*Δcolumn)

And compare (row, column) against the actual row_(Target0) andcolumn_(Target0). To move from one expected target to the next (e.g.from Target0 to Target1), we simply add Δrow and Δcolumn to row andcolumn respectively. To check if each target is on the line, we mustcalculate the expected position of Target0, and then perform one add andone comparison for each target ordinate.

At the end of comparing all 16 targets against a maximum of 90 lines,the result is the best line through the valid targets. If that linepasses through at least 8 targets (i.e. MaxFound>−8), it can be saidthat enough targets have been found to form a line, and thus the cardcan be processed. If the best line passes through fewer than 8, then thecard is considered invalid.

The resulting algorithm takes 180 divides to calculate Δrow and Δcolumn,180 multiply/adds to calculate target0 position, and then 2880adds/comparisons. The time we have to perform this processing is thetime taken to read 36 columns of pixel data=3,374,892 ns. Not evenaccounting for the fact that an add takes less time than a divide, it isnecessary to perform 3240 mathematical operations in 3,374,892 ns. Thatgives approximately 1040 ns per operation, or 104 cycles. The CPU cantherefore safely perform the entire processing of targets, reducingcomplexity of design.

Update Centroids Based on Data Edge Border and Clockmarks

Step 0: Locate the Data Area

From Target 0 (241 of FIG. 7) it is a predetermined fixed distance inrows and columns to the top left border 244 of the data area, and then afurther 1 dot column to the vertical clock marks 276. So we use TargetA,Δrow and Δcolumn found in the previous stage (Δrow and Δcolumn refer todistances between targets) to calculate the centroid or expectedlocation for Target0 as described previously.

Since the fixed pixel offset from Target0 to the data area is related tothe distance between targets (192 dots between targets, and 24 dotsbetween Target0 and the data area 243), simply add Δrow/8 to Target0'scentroid column coordinate (aspect ratio of dots is 1:1). Thus the topco-ordinate can be defined as:(column_(DotColumnTop)=column_(Target0)+(Δrow/8)(row_(DotColumnTop)=row_(Target)0+(Δcolumn/8)

Next Δrow and Δcolumn are updated to give the number of pixels betweendots in a single column (instead of between targets) by dividing them bythe number of dots between targets:Δrow=Δrow/192Δcolumn=Δcolumn/192

We also set the currentColumn register (see Phase 2) to be −1 so thatafter step 2, when phase 2 begins, the currentColumn register willincrement from −1 to 0.

Step 1: Write Out the Initial Centroid Deltas (Δ) and Bit History

This simply involves writing setup information required for Phase 2.

This can be achieved by writing 0s to all the Δrow and Δcolumn entriesfor each row, and a bit history. The bit history is actually an expectedbit history since it is known that to the left of the clock mark column276 is a border column 277, and before that, a white area. The bithistory therefore is 011, 010, 011, 010 etc.

Step 2: Update the Centroids Based on Actual Pixels Read.

The bit history is set up in Step 1 according to the expected clockmarks and data border. The actual centroids for each dot row can now bemore accurately set (they were initially 0) by comparing the expecteddata against the actual pixel values. The centroid updating mechanism isachieved by simply performing step 3 of Phase 2.

Phase 2—Detect Bit Pattern From Artcard Based on Pixels Read and Writeas Bytes.

Since a dot from the Artcard 9 requires a minimum of 9 sensed pixelsover 3 columns to be represented, there is little point in performingdot detection calculations every sensed pixel column. It is better toaverage the time required for processing over the average dotoccurrence, and thus make the most of the available processing time.This allows processing of a column of dots from an Artcard 9 in the timeit takes to read 3 columns of data from the Artcard. Although the mostlikely case is that it takes 4 columns to represent a dot, the 4^(th)column will be the last column of one dot and the first column of a nextdot. Processing should therefore be limited to only 3 columns.

As the pixels from the CCD are written to the DRAM in 13% of the timeavailable, 83% of the time is available for processing of 1 column ofdots i.e. 83% of (93,747*3)=83% of 281,241 ns=233,430 ns.

In the available time, it is necessary to detect 3150 dots, and writetheir bit values into the raw data area of memory. The processingtherefore requires the following steps:

For each column of dots on the Artcard:

Step 0: Advance to the next dot column

Step 1: Detect the top and bottom of an Artcard dot column (check clockmarks)

Step 2: Process the dot column, detecting bits and storing themappropriately

Step 3: Update the centroids

Since we are processing the Artcard's logical dot columns, and these mayshift over 165 pixels, the worst case is that we cannot process thefirst column until at least 165 columns have been read into DRAM. Phase2 would therefore finish the same amount of time after the read processhad terminated. The worst case time is: 165*93,747 ns=15,468,255 ns or0.015 seconds.

Step 0: Advance to the Next Dot Column

In order to advance to the next column of dots we add Δrow and Δcolumnto the dotColumnTop to give us the centroid of the dot at the top of thecolumn. The first time we do this, we are currently at the clock markscolumn 276 to the left of the bit image data area, and so we advance tothe first column of data. Since Δrow and Δcolumn refer to distancebetween dots within a column, to move between dot columns it isnecessary to add Δrow to column_(dotColumnTop) and Δcolumn torow_(dotColumnTop).

To keep track of what column number is being processed, the columnnumber is recorded in a register called CurrentColumn. Every time thesensor advances to the next dot column it is necessary to increment theCurrentColumn register. The first time it is incremented, it isincremented from −1 to 0 (see Step 0 Phase 1). The CurrentColumnregister determines when to terminate the read process (when reachingmaxColumns), and also is used to advance the DataOut Pointer to the nextcolumn of byte information once all 8 bits have been written to the byte(once every 8 dot columns). The lower 3 bits determine what bit we're upto within the current byte. It will be the same bit being written forthe whole column.

Step 1: Detect the Top and Bottom of an Artcard Dot Column.

In order to process a dot column from an Artcard, it is necessary todetect the top and bottom of a column. The column should form a straightline between the top and bottom of the column (except for local warpingetc.). Initially dotColumnTop points to the clock mark column 276. Wesimply toggle the expected value, write it out into the bit history, andmove on to step 2, whose first task will be to add the Δrow and Δcolumnvalues to dotColumnTop to arrive at the first data dot of the column.

Step 2: Process an Artcard's Dot Column

Given the centroids of the top and bottom of a column in pixelcoordinates the column should form a straight line between them, withpossible minor variances due to warping etc.

Assuming the processing is to start at the top of a column (at the topcentroid coordinate) and move down to the bottom of the column,subsequent expected dot centroids are given as:row_(next)=row+Δrowcolumn_(next)=column+Δcolumn

This gives us the address of the expected centroid for the next dot ofthe column. However to account for local warping and error we addanother Δrow and Δcolumn based on the last time we found the dot in agiven row. In this way we can account for small drifts that accumulateinto a maximum drift of some percentage from the straight line joiningthe top of the column to the bottom.

We therefore keep 2 values for each row, but store them in separatetables since the row history is used in step 3 of this phase.

-   -   Δrow and Δcolumn (2@4 bits each=1 byte)    -   row history (3 bits per row, 2 rows are stored per byte)

For each row we need to read a Δrow and Δcolumn to determine the changeto the centroid. The read process takes 5% of the bandwidth and 2 cachelines:76*(3150/32)+2*3150=13,824 ns=5% of bandwidth

Once the centroid has been determined, the pixels around the centroidneed to be examined to detect the status of the dot and hence the valueof the bit. In the worst case a dot covers a 4×4 pixel area. However,thanks to the fact that we are sampling at 3 times the resolution of thedot, the number of pixels required to detect the status of the dot andhence the bit value is much less than this. We only require access to 3columns of pixel columns at any one time.

In the worst case of pixel drift due to a 1% rotation, centroids willshift 1 column every 57 pixel rows, but since a dot is 3 pixels indiameter, a given column will be valid for 171 pixel rows (3*57). As abyte contains 2 pixels, the number of bytes valid in each buffered read(4 cache lines) will be a worst case of 86 (out of 128 read).

Once the bit has been detected it must be written out to DRAM. We storethe bits from 8 columns as a set of contiguous bytes to minimize DRAMdelay. Since all the bits from a given dot column will correspond to thenext bit position in a data byte, we can read the old value for thebyte, shift and OR in the new bit, and write the byte back.

The read/shift&OR/write process requires 2 cache lines.

We need to read and write the bit history for the given row as we updateit We only require 3 bits of history per row, allowing the storage of 2rows of history in a single byte. The read/shift&OR/write processrequires 2 cache lines.

The total bandwidth required for the bit detection and storage issummarised in the following table:

Read centroid Δ  5% Read 3 columns of pixel data 19% Read/Write detectedbits into byte buffer 10% Read/Write bit history  5% TOTAL 39%Detecting a Dot

The process of detecting the value of a dot (and hence the value of abit) given a centroid is accomplished by examining 3 pixel values andgetting the result from a lookup table. The process is fairly simple andis illustrated in FIG. 11. A dot 290 has a radius of about 1.5 pixels.Therefore the pixel 291 that holds the centroid, regardless of theactual position of the centroid within that pixel, should be 100% of thedot's value. If the centroid is exactly in the center of the pixel 291,then the pixels above 292 & below 293 the centroid's pixel, as well asthe pixels to the left 294 & right 295 of the centroid's pixel willcontain a majority of the dot's value. The further a centroid is awayfrom the exact center of the pixel 295, the more likely that more thanthe center pixel will have 100% coverage by the dot.

Although FIG. 11 only shows centroids differing to the left and belowthe center, the same relationship obviously holds for centroids aboveand to the right of center. center. In Case 1, the centroid is exactlyin the center of the middle pixel 295. The center pixel 295 iscompletely covered by the dot, and the pixels above, below, left, andright are also well covered by the dot. In Case 2, the centroid is tothe left of the center of the middle pixel 291. The center pixel isstill completely covered by the dot, and the pixel 294 to the left ofthe center is now completely covered by the dot. The pixels above 292and below 293 are still well covered. In Case 3, the centroid is belowthe center of the middle pixel 291. The center pixel 291 is stillcompletely covered by the dot 291, and the pixel below center is nowcompletely covered by the dot. The pixels left 294 and right 295 ofcenter are still well covered. In Case 4, the centroid is left and belowthe center of the middle pixel. The center pixel 291 is still completelycovered by the dot, and both the pixel to the left of center 294 and thepixel below center 293 are completely covered by the dot.

The algorithm for updating the centroid uses the distance of thecentroid from the center of the middle pixel 291 in order to select 3representative pixels and thus decide the value of the dot:

Pixel 1: the pixel containing the centroid

Pixel 2: the pixel to the left of Pixel 1 if the centroid's X coordinate(column value) is <½, otherwise the pixel to the right of Pixel 1.

Pixel 3: the pixel above pixel 1 if the centroid's Y coordinate (rowvalue) is <½, otherwise the pixel below Pixel 1.

As shown in FIG. 12, the value of each pixel is output to apre-calculated lookup table 301. The 3 pixels are fed into a 12-bitlookup table, which outputs a single bit indicating the value of thedot—on or off. The lookup table 301 is constructed at chip definitiontime, and can be compiled into about 500 gates. The lookup table can bea simple threshold table, with the exception that the center pixel(Pixel 1) is weighted more heavily.

Step 3: Update the Centroid Δs For Each Row in the Column

The idea of the Δs processing is to use the previous bit history togenerate a ‘perfect’ dot at the expected centroid location for each rowin a current column. The actual pixels (from the CCD) are compared withthe expected ‘perfect’ pixels. If the two match, then the actualcentroid location must be exactly in the expected position, so thecentroid Δs must be valid and not need updating. Otherwise a process ofchanging the centroid Δs needs to occur in order to best fit theexpected centroid location to the actual data. The new centroid Δs willbe used for processing the dot in the next column.

Updating the centroid Δs is done as a subsequent process from Step 2 forthe following reasons:

to reduce complexity in design, so that it can be performed as Step 2 ofPhase 1 there is enough bandwidth remaining to allow it to allow reuseof DRAM buffers, and

to ensure that all the data required for centroid updating is availableat the start of the process without special pipelining.

The centroid Δ are processed as Δcolumn Δrow respectively to reducecomplexity.

Although a given dot is 3 pixels in diameter, it is likely to occur in a4×4 pixel area. However the edge of one dot will as a result be in thesame pixel as the edge of the next dot. For this reason, centroidupdating requires more than simply the information about a given singledot.

FIG. 13 shows a single dot 310 from the previous column with a givencentroid 311. In this example, the dot 310 extend Δ over 4 pixel columns312–315 and in fact, part of the previous dot column's dot(coordinate=(Prevcolumn, Current Row)) has entered the current columnfor the dot on the current row. If the dot in the current row and columnwas white, we would expect the rightmost pixel column 314 from theprevious dot column to be a low value, since there is only the dotinformation from the previous column's dot (the current column's dot iswhite). From this we can see that the higher the pixel value is in thispixel column 315, the more the centroid should be to the right Ofcourse, if the dot to the right was also black, we cannot adjust thecentroid as we cannot get information sub-pixel. The same can be saidfor the dots to the left, above and below the dot at dot coordinates(PrevColumn, CurrentRow).

From this we can say that a maximum of 5 pixel columns and rows arerequired. It is possible to simplify the situation by taking the casesof row and column centroid Δs separately, treating them as the sameproblem, only rotated 90 degrees.

Taking the horizontal case first, it is necessary to change the columncentroid Δs if the expected pixels don't match the detected pixels. Fromthe bit history, the value of the bits found for the Current Row in thecurrent dot column, the previous dot column, and the (previous-1)th dotcolumn are known. The expected centroid location is also known. Usingthese two pieces of information, it is possible to generate a 20 bitexpected bit pattern should the read be ‘perfect’. The 20 bitbit-pattern represents the expected Δ values for each of the 5 pixelsacross the horizontal dimension. The first nibble would represent therightmost pixel of the leftmost dot. The next 3 nibbles represent the 3pixels across the center of the dot 310 from the previous column, andthe last nibble would be the leftmost pixel 317 of the rightmost dot(from the current column).

If the expected centroid is in the center of the pixel, we would expecta 20 bit pattern based on the following table:

Bit history Expected pixels 000 00000 001 0000D 010 0DFD0 011 0DFDD 100D0000 101 D000D 110 DDFD0 111 DDFDD

The pixels to the left and right of the center dot are either 0 or Ddepending on whether the bit was a 0 or 1 respectively. The center threepixels are either 000 or DFD depending on whether the bit was a 0 or 1respectively. These values are based on the physical area taken by a dotfor a given pixel. Depending on the distance of the centroid from theexact center of the pixel, we would expect data shifted slightly, whichreally only affects the pixels either side of the center pixel. Sincethere are 16 possibilities, it is possible to divide the distance fromthe center by 16 and use that amount to shift the expected pixels.

Once the 20 bit 5 pixel expected value has been determined it can becompared against the actual pixels read. This can proceed by subtractingthe expected pixels from the actual pixels read on a pixel by pixelbasis, and finally adding the differences together to obtain a distancefrom the expected Δ values.

FIG. 14 illustrates one form of implementation of the above algorithmwhich includes a look up table 320 which receives the bit history 322and central fractional component 323 and outputs 324 the corresponding20 bit number which is subtracted 321 from the central pixel input 326to produce a pixel difference 327.

This process is carried out for the expected centroid and once for ashift of the centroid left and right by 1 amount in Δcolumn. Thecentroid with the smallest difference from the actual pixels isconsidered to be the ‘winner’ and the Δcolumn updated accordingly (whichhopefully is ‘no change’). As a result, a Δcolumn cannot change by morethan 1 each dot column.

The process is repeated for the vertical pixels, and Δrow isconsequentially updated.

There is a large amount of scope here for parallelism. Depending on therate of the clock chosen for the ACP unit 31 these units can be placedin series (and thus the testing of 3 different Δ could occur inconsecutive clock cycles), or in parallel where all 3 can be testedsimultaneously. If the clock rate is fast enough, there is less need forparallelism.

Bandwidth Utilization

It is necessary to read the old Δ of the Δs, and to write them outagain. This takes 10% of the bandwidth:2*(76(3150/32)+2*3150)=27,648 ns=10% of bandwidth

It is necessary to read the bit history for the given row as we updateits Δs. Each byte contains 2 row's bit histories, thus taking 2.5% ofthe bandwidth:76((3150/2)/32)+2*(3150/2)=4,085 ns=2.5% of bandwidth

In the worst case of pixel drift due to a 1% rotation, centroids willshift 1 column every 57 pixel rows, but since a dot is 3 pixels indiameter, a given pixel column will be valid for 171 pixel rows (3*57).As a byte contains 2 pixels, the number of bytes valid in cached readswill be a worst case of 86 (out of 128 read). The worst case timing for5 columns is therefore 31% bandwidth.5*(((9450/(128*2))*320)*128/86)=88,112 ns=31% of bandwidth.

The total bandwidth required for the updating the centroid A issummarised in the following table:

Read/Write centroid Δ   10% Read bit history  2.5% Read 5 columns ofpixel data   31% TOTAL 43.5%Memory Usage For Phase 2:

The 2 MB bit-image DRAM area is read from and written to during Phase 2processing. The 2 MB pixel-data DRAM area is read.

The 0.5 MB scratch DRAM area is used for storing row data, namely:

Centroid array 24 bits (16:8) * 2 * 3150 = 18,900 byes Bit History array 3 bits * 3150 entries (2 per byte) = 1575 bytesPhase 3—Unscramble and XOR the Raw Data

Returning to FIG. 6, the next step in decoding is to unscramble and XORthe raw data. The 2 MB byte image, as taken from the Artcard, is in ascrambled XORed form. It must be unscrambled and re-XORed to retrievethe bit image necessary for the Reed Solomon decoder in phase 4.

Turning to FIG. 15, the unscrambling process 330 takes a 2 MB scrambledbyte image 331 and writes an unscrambled 2 MB image 332. The processcannot reasonably be performed in-place, so 2 sets of 2 MB areas areutilised. The scrambled data 331 is in symbol block order arranged in a16×16 array, with symbol block 0 (334) having all the symbol 0's fromall the code words in random order. Symbol block 1 has all the symbol1's from all the code words in random order etc. Since there are only255 symbols, the 256^(th) symbol block is currently unused.

A linear feedback shift register is used to determine the relationshipbetween the position within a symbol block eg. 334 and what code wordeg. 355 it came from. This works as long as the same seed is used whengenerating the original Artcard images. The XOR of bytes fromalternative source lines with 0×AA and 0×55 respectively is effectivelyfree (in time) since the bottleneck of time is waiting for the DRAM tobe ready to read/write to non-sequential addresses.

The timing of the unscrambling XOR process is effectively 2 MB of randombyte-reads, and 2 MB of random byte-writes i.e. 2*(2 MB*76 ns+2 MB*2ns)=327,155,712 ns or approximately 0.33 seconds. This timing assumes nocaching.

Phase 4—Reed Solomon Decode

This phase is a loop, iterating through copies of the data in the bitimage, passing them to the Reed-Solomon decode module until either asuccessful decode is made or until there are no more copies to attemptdecode from.

The Reed-Solomon decoder used can be the VLIW processor, suitablyprogrammed or, alternatively, a separate hardwired core such as LSILogic's L64712. The L64712 has a throughput of 50 Mbits per second(around 6.25 MB per second), so the time may be bound by the speed ofthe Reed-Solomon decoder rather than the 2 MB read and 1 MB write memoryaccess time (500 M/sec for sequential accesses). The time taken in theworst case is thus 2/6.25 s=approximately 0.32 seconds.

Phase 5 Running the Vark Script

The overall time taken to read the Artcard 9 and decode it is thereforeapproximately 2.15 seconds. The apparent delay to the user is actuallyonly 0.65 seconds (the total of Phases 3 and 4), since the Artcard stopsmoving after 1.5 seconds.

Once the Artcard is loaded, the Artvark script must be interpreted,Rather than run the script immediately, the script is only run upon thepressing of the ‘Print’ button 13 (FIG. 1). The time taken to run thescript will vary depending on the complexity of the script, and must betaken into account for the perceived delay between pressing the printbutton and the actual print button and the actual printing.

Alternative Artcard Format

Of course, other artcard formats are possible. There will now bedescribed one such alternative artcard format with a number ofpreferable feature. Described hereinafter will be the alternativeArtcard data format, a mechanism for mapping user data onto dots on analternative Artcard, and a fast alternative Artcard reading algorithmfor use in embedded systems where resources are scarce.

Alternative Artcard Overview

The Alternative Artcards can be used in both embedded and PC typeapplications, providing a user-friendly interface to large amounts ofdata or configuration information.

While the back side of an alternative Artcard has the same visualappearance regardless of the application (since it stores the data), thefront of an alternative Artcard can be application dependent. It mustmake sense to the user in the context of the application.

Alternative Artcard technology can also be independent of the printingresolution. The notion of storing data as dots on a card simply meansthat if it is possible put more dots in the same space (by increasingresolution), then those dots can represent more data. The preferredembodiment assumes utilisation of 1600 dpi printing on a 86 mm×55 mmcard as the sample Artcard, but it is simple to determine alternativeequivalent layouts and data sizes for other card sizes and/or otherprint resolutions. Regardless of the print resolution, the readingtechnique remain the same. After all decoding and other overhead hasbeen taken into account, alternative Artcards are capable of storing upto 1 Megabyte of data at print resolutions up to 1600 dpi. AlternativeArtcards can store megabytes of data at print resolutions greater than1600 dpi. The following two tables summarize the effective alternativeArtcard data storage capacity for certain print resolutions:

Format of an Alternative Artcard

The structure of data on the alternative Artcard is thereforespecifically designed to aid the recovery of data. This sectiondescribes the format of the data (back) side of an alternative Artcard.

Dots

The dots on the data side of an alternative Artcard can be monochrome.For example, black dots printed on a white background at a predetermineddesired print resolution. Consequently a “black dot” is physicallydifferent from a “white dot”. FIG. 16 illustrates various examples ofmagnified views of black and white dots. The monochromatic scheme ofblack dots on a white background is preferably chosen to maximizedynamic range in blurry reading environments. Although the black dotsare printed at a particular pitch (eg. 1600 dpi), the dots themselvesare slightly larger in order to create continuous lines when dots areprinted contiguously. In the example images of FIG. 16, the dots are notas merged as they may be in reality as a result of bleeding. There wouldbe more smoothing out of the black indentations. Although thealternative Artcard system described in the preferred embodiment allowsfor flexibly different dot sizes, exact dot sizes and ink/printingbehaviour for a particular printing technology should be studied in moredetail in order to obtain best results.

In describing this artcard embodiment, the term dot refers to a physicalprinted dot (ink, thermal, electro-photographic, silver-halide etc) onan alternative Artcard. When an alternative Artcard reader scans analternative Artcard, the dots must be sampled at least double theprinted resolution to satisfy Nyquist's Theorem. The term pixel refersto a sample value from an alternative Artcard reader device. Forexample, when 1600 dpi dots are scanned at 4800 dpi there are 3 pixelsin each dimension of a dot, or 9 pixels per dot. The sampling processwill be further explained hereinafter.

Turning to FIG. 17, there is shown the data surface 1101 a sample ofalternative Artcard. Each alternative Artcard consists of an “active”region 1102 surrounded by a white border region 1103. The white border1103 contains no data information, but can be used by an alternativeArtcard reader to calibrate white levels. The active region is an arrayof data blocks eg. 1104, with each data block separated from the next bya gap of 8 white dots eg. 1106. Depending on the print resolution, thenumber of data blocks on an alternative Artcard will vary. On a 1600 dpialternative Artcard, the array can be 8×8. Each data block 1104 hasdimensions of 627×394 dots. With an inter-block gap 1106 of 8 whitedots, the active area of an alternative Artcard is therefore 5072×3208dots (8.1 mm×5.1 mm at 1600 dpi).

Data Blocks

Turning now to FIG. 18, there is shown a single data block 1107. Theactive region of an alternative Artcard consists of an array ofidentically structured data blocks 1107. Each of the data blocks has thefollowing structure: a data region 1108 surrounded by clock-marks 1109,borders 1110, and targets 1111. The data region holds the encoded dataproper, while the clock-marks, borders and targets are presentspecifically to help locate the data region and ensure accurate recoveryof data from within the region.

Each data block 1107 has dimensions of 627×394 dots. Of this, thecentral area of 595×384 dots is the data region 1108. The surroundingdots are used to hold the clock-marks, borders, and targets.

Borders and Clockmarks

FIG. 19 illustrates a data block with FIG. 20 and FIG. 21 illustratingmagnified edge portions thereof. As illustrated in FIG. 20 and FIG. 21,there are two 5 dot high border and clockmark regions 1170, 1177 in eachdata block: one above and one below the data region. For example, Thetop 5 dot high region consists of an outer black dot border line 1112(which stretches the length of the data block), a white dot separatorline 1113 (to ensure the border line is independent), and a 3 dot highset of clock marks 1114. The clock marks alternate between a white andblack row, starting with a black clock mark at the 8th column fromeither end of the data block. There is no separation between clockmarkdots and dots in the data region.

The clock marks are symmetric in that if the alternative Artcard isinserted rotated 180 degrees, the same relative border/clockmark regionswill be encountered. The border 1112, 1113 is intended for use by analternative Artcard reader to keep vertical tracking as data is readfrom the data region. The clockmarks 1114 are intended to keephorizontal tracking as data is read from the data region. The separationbetween the border and clockmarks by a white line of dots is desirableas a result of blurring occurring during reading. The border thusbecomes a black line with white on either side, making for a goodfrequency response on reading. The clockmarks alternating between whiteand black have a similar result, except in the horizontal rather thanthe vertical dimension. Any alternative Artcard reader must locate theclockmarks and border if it intends to use them for tracking. The nextsection deals with targets, which are designed to point the way to theclockmarks, border and data.

Targets in the Target Region

As shown in FIG. 23, there are two 15-dot wide target regions 1116, 1117in each data block: one to the left and one to the right of the dataregion. The target regions are separated from the data region by asingle column of dots used for orientation. The purpose of the TargetRegions 1116, 1117 is to point the way to the clockmarks, border anddata regions. Each Target Region contains 6 targets eg. 1118 that aredesigned to be easy to find by an alternative Artcard reader. Turningnow to FIG. 22 there is shown the structure of a single target 1120.Each target 1120 is a 15×15 dot black square with a center structure1121 and a run-length encoded target number 1122. The center structure1121 is a simple white cross, and the target number component 1122 issimply two columns of white dots, each being 2 dots long for each partof the target number. Thus target number 1's target id 1122 is 2 dotslong, target number 2's target id 1122 is 4 dots wide etc.

As shown in FIG. 23, the targets are arranged so that they are rotationinvariant with regards to card insertion. This means that the lefttargets and right targets are the same, except rotated 180 degrees. Inthe left Target Region 1116, the targets are arranged such that targets1 to 6 are located top to bottom respectively. In the right TargetRegion, the targets are arranged so that target numbers 1 to 6 arelocated bottom to top. The target number id is always in the halfclosest to the data region. The magnified view portions of FIG. 23reveals clearly the how the right targets are simply the same as theleft targets, except rotated 180 degrees.

As shown in FIG. 24, the targets 1124, 1125 are specifically placedwithin the Target Region with centers 55 dots apart. In addition, thereis a distance of 55 dots from the center of target 1 (1124) to the firstclockmark dot 1126 in the upper clockmark region, and a distance of 55dots from the center of the target to the first clockmark dot in thelower clockmark region (not shown). The first black clockmark in bothregions begins directly in line with the target center (the 8th dotposition is the center of the 15 dot-wide target).

The simplified schematic illustrations of FIG. 24 illustrates thedistances between target centers as well as the distance from Target 1(1124) to the first dot of the first black clockmark (1126) in the upperborder/clockmark region. Since there is a distance of 55 dots to theclockmarks from both the upper and lower targets, and both sides of thealternative Artcard are symmetrical (rotated through 180 degrees), thecard can be read left-to-right or right-to-left. Regardless of readingdirection, the orientation does need to be determined in order toextract the data from the data region.

Orientation Columns

As illustrated in FIG. 25, there are two 1 dot wide Orientation Columns1127, 1128 in each data block: one directly to the left and one directlyto the right of the data region. The Orientation Columns are present togive orientation information to an alternative Artcard reader On theleft side of the data region (to the right of the Left Targets) is asingle column of white dots 1127. On the right side of the data region(to the left of the Right Targets) is a single column of black dots1128. Since the targets are rotation invariant, these two columns ofdots allow an alternative Artcard reader to determine the orientation ofthe alternative Artcard—has the card been inserted the right way, orback to front. From the alternative Artcard reader's point of view,assuming no degradation to the dots, there are two possibilities:

-   -   If the column of dots to the left of the data region is white,        and the column to the right of the data region is black, then        the reader will know that the card has been inserted the same        way as it was written.    -   If the column of dots to the left of the data region is black,        and the column to the right of the data region is white, then        the reader will know that the card has been inserted backwards,        and the data region is appropriately rotated. The reader must        take appropriate action to correctly recover the information        from the alternative Artcard.        Data Region

As shown in FIG. 26, the data region of a data block consists of 595columns of 384 dots each, for a total of 228,480 dots. These dots mustbe interpreted and decoded to yield the original data. Each dotrepresents a single bit, so the 228,480 dots represent 228,480 bits, or28,560 bytes. The interpretation of each dot can be as follows:

Black 1 White 0

The actual interpretation of the bits derived from the dots, however,requires understanding of the mapping from the original data to the dotsin the data regions of the alternative Artcard.

Mapping Original Data to Data Region Dots

There will now be described the process of taking an original data fileof maximum size 910,082 bytes and mapping it to the dots in the dataregions of the 64 data blocks on a 1600 dpi alternative Artcard. Analternative Artcard reader would reverse the process in order to extractthe original data from the dots on an alternative Artcard. At firstglance it seems trivial to map data onto dots: binary data is comprisedof 1s and 0s, so it would be possible to simply write black and whitedots onto the card. This scheme however, does not allow for the factthat ink can fade, parts of a card may be damaged with dirt, grime, oreven scratches. Without error-detection encoding, there is no way todetect if the data retrieved from the card is correct. And withoutredundancy encoding, there is no way to correct the detected errors. Theaim of the mapping process then, is to make the data recovery highlyrobust, and also give the alternative Artcard reader the ability to knowit read the data correctly.

There are three basic steps involved in mapping an original data file todata region dots:

-   -   Redundancy encode the original data    -   Shuffle the encoded data in a deterministic way to reduce the        effect of localized alternative Artcard damage    -   Write out the shuffled, encoded data as dots to the data blocks        on the alternative Artcard

Each of these steps is examined in detail in the following sections.

Redundancy Encode Using Reed-Solomon Encoding

The mapping of data to alternative Artcard dots relies heavily on themethod of redundancy encoding employed. Reed-Solomon encoding ispreferably chosen for its ability to deal with burst errors andeffectively detect and correct errors using a minimum of redundancy.Reed Solomon encoding is adequately discussed in the standard texts suchas Wicker, S., and Bhargava, V., 1994, Reed-Solomon Codes and theirApplications, IEEE Press. Rorabaugh, C, 1996, Error Coding Cookbook,McGraw-Hill. Lyppens, H., 1997, Reed-Solomon Error Correction, Dr.Dobb's Journal, January 1997 (Volume 22, Issue 1).

A variety of different parameters for Reed-Solomon encoding can be used,including different symbol sizes and different levels of redundancy.Preferably, the following encoding parameters are used:

-   -   m=8    -   t=64

Having m=8 means that the symbol size is 8 bits (1 byte). It also meansthat each Reed-Solomon encoded block size n is 255 bytes (2⁸−1 symbols).In order to allow correction of up to t symbols, 2 t symbols in thefinal block size must be taken up with redundancy symbols. Having t=64means that 64 bytes (symbols) can be corrected per block if they are inerror. Each 255 byte block therefore has 128 (2×64) redundancy bytes,and the remaining 127 bytes (k=127) are used to hold original data.Thus:

-   -   n=255    -   k=127

The practical result is that 127 bytes of original data are encoded tobecome a 255-byte block of Reed-Solomon encoded data. The encoded255-byte blocks are stored on the alternative Artcard and later decodedback to the original 127 bytes again by the alternative Artcard reader.The 384 dots in a single column of a data block's data region can hold48 bytes (384/8). 595 of these columns can hold 28,560 bytes. Thisamounts to 112 Reed-Solomon blocks (each block having 255 bytes). The 64data blocks of a complete alternative Artcard can hold a total of 7168Reed-Solomon blocks (1,827,840 bytes, at 255 bytes per Reed-Solomonblock). Two of the 7,168 Reed-Solomon blocks are reserved for controlinformation, but the remaining 7166 are used to store data. Since eachReed-Solomon block holds 127 bytes of actual data, the total amount ofdata that can be stored on an alternative Artcard is 910,082 bytes(7166×127). If the original data is less than this amount, the data canbe encoded to fit an exact number of Reed-Solomon blocks, and then theencoded blocks can be replicated until all 7,166 are used. FIG. 27illustrates the overall form of encoding utilised.

Each of the 2 Control blocks 1132, 1133 contain the same encodedinformation required for decoding the remaining 7,166 Reed-Solomonblocks:

The number of Reed-Solomon blocks in a full message (16 bits storedlo/hi), and

The number of data bytes in the last Reed-Solomon block of the message(8 bits)

These two numbers are repeated 32 times (consuming. 96 bytes) with theremaining 31 bytes reserved and set to 0. Each control block is thenReed-Solomon encoded, turning the 127 bytes of control information into255 bytes of Reed-Solomon encoded data.

The Control Block is stored twice to give greater chance of itsurviving. In addition, the repetition of the data within the ControlBlock has particular significance when using Reed-Solomon encoding. Inan uncorrupted Reed-Solomon encoded block, the first 127 bytes of dataare exactly the original data, and can be looked at in an attempt torecover the original message if the Control Block fails decoding (morethan 64 symbols are corrupted). Thus, if a Control Block fails decoding,it is possible to examine sets of 3 bytes in an effort to determine themost likely values for the 2 decoding parameters. It is not guaranteedto be recoverable, but it has a better chance through redundancy. Saythe last 159 bytes of the Control Block are destroyed, and the first 96bytes are perfectly ok. Looking at the first 96 bytes will show arepeating set of numbers. These numbers can be sensibly used to decodethe remainder of the message in the remaining 7,166 Reed-Solomon blocks.

By way of example, assume a data file containing exactly 9,967 bytes ofdata. The number of Reed-Solomon blocks required is 79. The first 78Reed-Solomon blocks are completely utilized, consuming 9,906 bytes(78×127). The 79th block has only 61 bytes of data (with the remaining66 bytes all 0s).

The alternative Artcard would consist of 7,168 Reed-Solomon blocks. Thefirst 2 blocks would be Control Blocks, the next 79 would be the encodeddata, the next 79 would be a duplicate of the encoded data, the next 79would be another duplicate of the encoded data, and so on. After storingthe 79 Reed-Solomon blocks 90 times, the remaining 56 Reed-Solomonblocks would be another duplicate of the first 56 blocks from the 79blocks of encoded data (the final 23 blocks of encoded data would not bestored again as there is not enough room on the alternative Artcard). Ahex representation of the 127 bytes in each Control Block data beforebeing Reed-Solomon encoded would be as illustrated in FIG. 28.

Scramble the Encoded Data

Assuming all the encoded blocks have been stored contiguously in memory,a maximum 1,827,840 bytes of data can be stored on the alternativeArtcard (2 Control Blocks and 7,166 information blocks, totalling 7,168Reed-Solomon encoded blocks). Preferably, the data is not directlystored onto the alternative Artcard at this stage however, or all 255bytes of one Reed-Solomon block will be physically together on the card.Any dirt, grime, or stain that causes physical damage to the card hasthe potential of damaging more than 64 bytes in a single Reed-Solomonblock, which would make that block unrecoverable. If there are noduplicates of that Reed-Solomon block, then the entire alternativeArtcard cannot be decoded.

The solution is to take advantage of the fact that there are a largenumber of bytes on the alternative Artcard, and that the alternativeArtcard has a reasonable physical size. The data can therefore bescrambled to ensure that symbols from a single Reed-Solomon block arenot in close proximity to one another. Of course pathological cases ofcard degradation can cause Reed-Solomon blocks to be unrecoverable, buton average, the scrambling of data makes the card much more robust. Thescrambling scheme chosen is simple and is illustrated schematically inFIG. 29. All the Byte 0s from each Reed-Solomon block are placedtogether 1136, then all the Byte 1s etc. There will therefore be 7,168byte 0's, then 7,168 Byte 1's etc. Each data block on the alternativeArtcard can store 28,560 bytes. Consequently there are approximately 4bytes from each Reed-Solomon block in each of the 64 data blocks on thealternative Artcard.

Under this scrambling scheme, complete damage to 16 entire data blockson the alternative Artcard will result in 64 symbol errors perReed-Solomon block. This means that if there is no other damage to thealternative Artcard, the entire data is completely recoverable, even ifthere is no data duplication.

Write the Scrambled Encoded Data to the Alternative Artcard

Once the original data has been Reed-Solomon encoded, duplicated, andscrambled, there are 1,827,840 bytes of data to be stored on thealternative Artcard. Each of the 64 data blocks on the alternativeArtcard stores 28,560 bytes.

The data is simply written out to the alternative Artcard data blocks sothat the first data block contains the first 28,560 bytes of thescrambled data, the second data block contains the next 28,560 bytesetc.

As illustrated in FIG. 30, within a data block, the data is written outcolumn-wise left to right. Thus the left-most column within a data blockcontains the first 48 bytes of the 28,560 bytes of scrambled data, andthe last column contains the last 48 bytes of the 28,560 bytes ofscrambled data. Within a column, bytes are written out top to bottom,one bit at a time, starting from bit 7 and finishing with bit 0. If thebit is set (1), a black dot is placed on the alternative Artcard, if thebit is clear (0), no dot is placed, leaving it the white backgroundcolor of the card.

For example, a set of 1,827,840 bytes of data can be created byscrambling 7,168 Reed-Solomon encoded blocks to be stored onto analternative Artcard. The first 28,560 bytes of data are written to thefirst data block. The first 48 bytes of the first 28,560 bytes arewritten to the first column of the data block, the next 48 bytes to thenext column and so on. Suppose the first two bytes of the 28,560 bytesare hex D3 5F. Those first two bytes will be stored in column 0 of thedata block. Bit 7 of byte 0 will be stored first, then bit 6 and so on.Then Bit 7 of byte 1 will be stored through to bit 0 of byte 1. Sinceeach “1” is stored as a black dot, and each “0” as a white dot, thesetwo bytes will be represented on the alternative Artcard as thefollowing set of dots:

-   -   D3 (1101 0011) becomes: black, black, white, black, white,        white, black, black    -   5F (0101 1111) becomes: white, black, white, black, black,        black, black, black        Decoding an Alternative Artcard

This section deals with extracting the original data from an alternativeArtcard in an accurate and robust manner. Specifically, it assumes thealternative Artcard format as described in the previous chapter, anddescribes a method of extracting the original pre-encoded data from thealternative Artcard.

There are a number of general considerations that are part of theassumptions for decoding an alternative Artcard.

User

The purpose of an alternative Artcard is to store data for use indifferent applications. A user inserts an alternative Artcard into analternative Artcard reader, and expects the data to be loaded in a“reasonable time”. From the user's perspective, a motor transport movesthe alternative Artcard into an alternative Artcard reader. This is notperceived as a problematic delay, since the alternative Artcard is inmotion. Any time after the alternative Artcard has stopped is perceivedas a delay, and should be minimized in any alternative Artcard readingscheme. Ideally, the entire alternative Artcard would be read while inmotion, and thus there would be no perceived delay after the card hadstopped moving.

For the purpose of the preferred embodiment, a reasonable time for analternative Artcard to be physically loaded is defined to be 1.5seconds. There should be a minimization of time for additional decodingafter the alternative Artcard has stopped moving. Since the Activeregion of an alternative Artcard covers most of the alternative Artcardsurface we can limit our timing concerns to that region.

Sampling Dots

The dots on an alternative Artcard must be sampled by a CCD reader orthe like at least at double the printed resolution to satisfy Nyquist'sTheorem. In practice it is better to sample at a higher rate than this.In the alternative Artcard reader environment, dots are preferablysampled at 3 times their printed resolution in each dimension, requiring9 pixels to define a single dot. If the resolution of the alternativeArtcard dots is 1600 dpi, the alternative Artcard reader's image sensormust scan pixels at 4800 dpi. Of course if a dot is not exactly alignedwith the sampling sensor, the worst and most likely case as illustratedin FIG. 31, is that a dot will be sensed over a 4×4 pixel area.

Each sampled pixel is 1 byte (8 bits). The lowest 2 bits of each pixelcan contain significant noise. Decoding algorithms must therefore benoise tolerant.

Alignment/Rotation

It is extremely unlikely that a user will insert an alternative Artcardinto an alternative Artcard reader perfectly aligned with no rotation.Certain physical constraints at a reader entrance and motor transportgrips will help ensure that once inserted, an alternative Artcard willstay at the original angle of insertion relative to the CCD. Preferablythis angle of rotation, as illustrated in FIG. 32 is a maximum of 1degree. There can be some slight aberrations in angle due to jitter andmotor rumble during the reading process, but these are assumed toessentially stay within the 1-degree limit.

The physical dimensions of an alternative Artcard are 86 mm×55 mm. A 1degree rotation adds 1.5 mm to the effective height of the card as 86 mmpasses under the CCD (86 sin 1°), which will affect the required CCDlength.

The effect of a 1 degree rotation on alternative Artcard reading is thata single scanline from the CCD will include a number of differentcolumns of dots from the alternative Artcard. This is illustrated in anexaggerated form in FIG. 32 which shows the drift of dots across thecolumns of pixels. Although exaggerated in this diagram, the actualdrift will be a maximum 1 pixel column shift every 57 pixels.

When an alternative Artcard is not rotated, a single column of dots canbe read over 3 pixel scanlines. The more an alternative Artcard isrotated, the greater the local effect. The more dots being read, thelonger the rotation effect is applied. As either of these factorsincrease, the larger the number of pixel scanlines that are needed to beread to yield a given set of dots from a single column on an alternativeArtcard. The following table shows how many pixel scanlines are requiredfor a single column of dots in a particular alternative Artcardstructure.

Region Height 0° rotation 1° rotation Active region 3208 dots 3 pixelcolumns 168 pixel columns Data block  394 dots 3 pixel columns  21 pixelcolumns

To read an entire alternative Artcard, we need to read 87 mm (86 mm+1 mmdue to 1° rotation). At 4800 dpi this implies 16,252 pixel columns.

CCD (or Other Linear Image Sensor) Length

The length of the CCD itself must accommodate:

-   -   the physical height of the alternative Artcard (55 mm),    -   vertical slop on physical alternative Artcard insertion (1 mm)    -   insertion rotation of up to 1 degree (86 sin 1°=1.5 mm)

These factors combine to form a total length of 57.5 mm.

When the alternative Artcard Image sensor CCD in an alternative Artcardreader scans at 4800 dpi, a single scanline is 10,866 pixels. Forsimplicity, this figure has been rounded up to 11,000 pixels. The ActiveRegion of an alternative Artcard has a height of 3208 dots, whichimplies 9,624 pixels. A Data Region has a height of 384 dots, whichimplies 1,152 pixels.

DRAM Size

The amount of memory required for alternative Artcard reading anddecoding is ideally minimized. The typical placement of an alternativeArtcard reader is an embedded system where memory resources areprecious. This is made more problematic by the effects of rotation. Asdescribed above, the more an alternative Artcard is rotated, the morescanlines are required to effectively recover original dots.

There is a trade-off between algorithmic complexity, user perceiveddelays, robustness, and memory usage. One of the simplest readeralgorithms would be to simply scan the whole alternative Artcard, andthen to process the whole data without real-time constraints. Not onlywould this require huge reserves of memory, it would take longer than areader algorithm that occurred concurrently with the alternative Artcardreading process.

The actual amount of memory required for reading and decoding analternative Artcard is twice the amount of space required to hold theencoded data, together with a small amount of scratch space (1–2 KB).For the 1600 dpi alternative Artcard, this implies a 4 MB memoryrequirement. The actual usage of the memory is detailed in the followingalgorithm description.

Transfer Rate

DRAM bandwidth assumptions need to be made for timing considerations andto a certain extent affect algorithmic design, especially sincealternative Artcard readers are typically part of an embedded system.

A standard Rambus Direct RDRAM architecture is assumed, as defined inRambus Inc, October 1997, Direct Rambus Technology Disclosure, with apeak data transfer rate of 1.6 GB/sec. Assuming 75% efficiency (easilyachieved), we have an average of 1.2 GB/sec data transfer rate. Theaverage time to access a block of 16 bytes is therefore 12 ns.

Dirty Data

Physically damaged alternative Artcards can be inserted into a reader.Alternative Artcards may be scratched, or be stained with grime or dirtA alternative Artcard reader can't assume to read everything perfectly.The effect of dirty data is made worse by blurring, as the dirty dataaffects the surrounding clean dots.

Blurry Environment

There are two ways that blurring is introduced into the alternativeArtcard reading environment:

-   -   Natural blurring due to nature of the CCD's distance from the        alternative Artcard.    -   Warping of alternative Artcard

Natural blurring of an alternative Artcard image occurs when there isoverlap of sensed data from the CCD. Blurring can be useful, as theoverlap ensures there are no high frequencies in the sensed data, andthat there is no data missed by the CCD. However if the area covered bya CCD pixel is too large, there will be too much blurring and thesampling required to recover the data will not be met. FIG. 33 is aschematic illustration of the overlapping of sensed data.

Another form of blurring occurs when an alternative Artcard is slightlywarped due to heat damage. When the warping is in the verticaldimension, the distance between the alternative Artcard and the CCD willnot be constant, and the level of blurring will vary across those areas.

Black and white dots were chosen for alternative Artcards to give thebest dynamic range in blurry reading environments. Blurring can causeproblems in attempting to determine whether a given dot is black orwhite.

As the blurring increases, the more a given dot is influenced by thesurrounding dots. Consequently the dynamic range for a particular dotdecreases. Consider a white dot and a black dot, each surrounded by allpossible sets of dots. The 9 dots are blurred, and the center dotsampled. FIG. 34 shows the distribution of resultant center dot valuesfor black and white dots.

The diagram is intended to be a representative blurring. The curve 1140from 0 to around 180 shows the range of black dots. The curve 1141 from75 to 250 shows the range of white dots. However the greater theblurring, the more the two curves shift towards the center of the rangeand therefore the greater the intersection area, which means the moredifficult it is to determine whether a given dot is black or white. Apixel value at the center point of intersection is ambiguous—the dot isequally likely to be a black or a white.

As the blurring increases, the likelihood of a read bit error increases.Fortunately, the Reed-Solomon decoding algorithm can cope with thesegracefully up to t symbol errors. FIG. 34 is a graph of number predictednumber of alternative Artcard Reed-Solomon blocks that cannot berecovered given a particular symbol error rate. Notice how theReed-Solomon decoding scheme performs well and then substantiallydegrades. If there is no Reed-Solomon block duplication, then only 1block needs to be in error for the data to be unrecoverable. Of course,with block duplication the chance of an alternative Artcard decodingincreases.

FIG. 35 only illustrates the symbol (byte) errors corresponding to thenumber of Reed-Solomon blocks in error. There is a trade-off between theamount of blurring that can be coped with, compared to the amount ofdamage that has been done to a card. Since all error detection andcorrection is performed by a Reed-Solomon decoder, there is a finitenumber of errors per Reed-Solomon data block that can be coped with. Themore errors introduced through blurring, the fewer the number of errorsthat can be coped with due to alternative Artcard damage.

Overview of Alternative Artcard Decoding

As noted previously, when the user inserts an alternative Artcard intoan alternative Artcard reading unit, a motor transport ideally carriesthe alternative Artcard past a monochrome linear CCD image sensor. Thecard is sampled in each dimension at three times the printed resolution.Alternative Artcard reading hardware and software compensate forrotation up to 1 degree, jitter and vibration due to the motortransport, and blurring due to variations in alternative Artcard to CCDdistance. A digital bit image of the data is extracted from the sampledimage by a complex method described here. Reed-Solomon decoding correctsarbitrarily distributed data corruption of up to 25% of the raw data onthe alternative Artcard. Approximately 1 MB of corrected data isextracted from a 1600 dpi card.

The steps involved in decoding are so as indicated in FIG. 36.

The decoding process requires the following steps:

-   -   Scan 1144 the alternative Artcard at three times printed        resolution (eg scan 1600 dpi alternative Artcard at 4800 dpi)    -   Extract 1145 the data bitmap from the scanned dots on the card.    -   Reverse 1146 the bitmap if the alternative Artcard was inserted        backwards.    -   Unscramble 1147 the encoded data    -   Reed-Solomon 1148 decode the data from the bitmap        Algorithmic Overview        Phase 1—Real Time Bit Image Extraction

A simple comparison between the available memory (4 MB) and the memoryrequired to hold all the scanned pixels for a 1600 dpi alternativeArtcard (172.5 MB) shows that unless the card is read multiple times(not a realistic option), the extraction of the bitmap from the pixeldata must be done on the fly, in real time, while the alternativeArtcard is moving past the CCD. Two tasks must be accomplished in thisphase:

-   -   Scan the alternative Artcard at 4800 dpi    -   Extract the data bitmap from the scanned dots on the card

The rotation and unscrambling of the bit image cannot occur until thewhole bit image has been extracted. It is therefore necessary to assigna memory region to hold the extracted bit image. The bit image fitseasily within 2 MB, leaving 2 MB for use in the extraction process.

Rather than extracting the bit image while looking only at the currentscanline of pixels from the CCD, it is possible to allocate a buffer toact as a window onto the alternative Artcard, storing the last Nscanlines read. Memory requirements do not allow the entire alternativeArtcard to be stored this way (172.5 MB would be required), butallocating 2 MB to store 190 pixel columns (each scanline takes lessthan 11,000 bytes) makes the bit image extraction process simpler.

The 4 MB memory is therefore used as follows:

-   -   2 MB for the extracted bit image    -   ˜2 MB for the scanned pixels    -   1.5 KB for Phase 1 scratch data (as required by algorithm)

The time taken for Phase 1 is 1.5 seconds, since this is the time takenfor the alternative Artcard to travel past the CCD and physically load.

Phase 2—Data Extraction From Bit Image

Once the bit image has been extracted, it must be unscrambled andpotentially rotated 180°. It must then be decoded. Phase 2 has noreal-time requirements, in that the alternative Artcard has stoppedmoving, and we are only concerned with the user's perception of elapsedtime. Phase 2 therefore involves the remaining tasks of decoding analternative Artcard:

-   -   Re-organize the bit image, reversing it if the alternative        Artcard was inserted backwards    -   Unscramble the encoded data    -   Reed-Solomon decode the data from the bit image

The input to Phase 2 is the 2 MB bit image buffer. Unscrambling androtating cannot be performed in situ, so a second 2 MB buffer isrequired. The 2 MB buffer used to hold scanned pixels in Phase 1 is nolonger required and can be used to store the rotated unscrambled data.

The Reed-Solomon decoding task takes the unscrambled bit image anddecodes it to 910,082 bytes. The decoding can be performed in situ, orto a specified location elsewhere. The decoding process does not requireany additional memory buffers.

The 4 MB memory is therefore used as follows:

-   -   2 MB for the extracted bit image (from Phase 1)    -   ˜2 MB for the unscrambled, potentially rotated bit image    -   <1 KB for Phase 2 scratch data (as required by algorithm)

The time taken for Phase 2 is hardware dependent and is bound by thetime taken for Reed-Solomon decoding. Using a dedicated core such as LSILogic's L64712, or an equivalent CPU/DSP combination, it is estimatedthat Phase 2 would take 0.32 seconds.

Phase 1—Extract Bit Image

This is the real-time phase of the algorithm, and is concerned withextracting the bit image from the alternative Artcard as scanned by theCCD.

As shown in FIG. 37 Phase 1 can be divided into 2 asynchronous processstreams. The first of these streams is simply the real-time reader ofalternative Artcard pixels from the CCD, writing the pixels to DRAM. Thesecond stream involves looking at the pixels, and extracting the bits.The second process stream is itself divided into 2 processes. The firstprocess is a global process, concerned with locating the start of thealternative Artcard. The second process is the bit image extractionproper.

FIG. 38 illustrates the data flow from a data/process perspective.

Timing

For an entire 1600 dpi alternative Artcard, it is necessary to read amaximum of 16,252 pixel-columns. Given a total time of 1.5 seconds forthe whole alternative Artcard, this implies a maximum time of 92,296 nsper pixel column during the course of the various processes.

Process 1—Read Pixels from CCD

The CCD scans the alternative Artcard at 4800 dpi, and generates 11,0001-byte pixel samples per column. This process simply takes the data fromthe CCD and writes it to DRAM, completely independently of any otherprocess that is reading the pixel data from DRAM. FIG. 39 illustratesthe steps involved.

The pixels are written contiguously to a 2 MB buffer that can hold 190full columns of pixels. The buffer always holds the 190 columns mostrecently read. Consequently, any process that wants to read the pixeldata (such as Processes 2 and 3) must firstly know where to look for agiven column, and secondly, be fast enough to ensure that the datarequired is actually in the buffer.

Process 1 makes the current scanline number (CurrentScanLine) availableto other processes so they can ensure they are not attempting to accesspixels from scanlines that have not been read yet.

The time taken to write out a single column of data (11,000 bytes) toDRAM is:11,000/16*12=8,256 ns

Process 1 therefore uses just under 9% of the available DRAM bandwidth(8256/92296).

Process 2—Detect Start of Alternative Artcard

This process is concerned with locating the Active Area on a scannedalternative Artcard. The input to this stage is the pixel data from DRAM(placed there by Process 1). The output is a set of bounds for the first8 data blocks on the alternative Artcard, required as input to Process3. A high level overview of the process can be seen in FIG. 40.

An alternative Artcard can have vertical slop of 1 mm upon insertion.With a rotation of 1 degree there is further vertical slop of 1.5 mm (86sin 1°). Consequently there is a total vertical slop of 2.5 mm. At 1600dpi, this equates to a slop of approximately 160 dots. Since a singledata block is only 394 dots high, the slop is just under half a datablock. To get a better estimate of where the data blocks are located thealternative Artcard itself needs to be detected.

Process 2 therefore consists of two parts:

-   -   Locate the start of the alternative Artcard, and if found,    -   Calculate the bounds of the first 8 data blocks based on the        start of the alternative Artcard.        Locate the Start of the Alternative Artcard

The scanned pixels outside the alternative Artcard area are black (thesurface can be black plastic or some other non-reflective surface). Theborder of the alternative Artcard area is white. If we process the pixelcolumns one by one, and filter the pixels to either black or white, thetransition point from black to white will mark the start of thealternative Artcard. The highest level process is as follows:

for (Column=0; Column < MAX_COLUMN; Column++) {   Pixel =ProcessColumn(Column)   if (Pixel)       return (Pixel, Column)    //success! } return failure           // no alternative Artcard found

The ProcessColumn function is simple. Pixels from two areas of thescanned column are passed through a threshold filter to determine ifthey are black or white. It is possible to then wait for a certainnumber of white pixels and announce the start of the alternative Artcardonce the given number has been detected. The logic of processing a pixelcolumn is shown in the following pseudocode. 0 is returned if thealternative Artcard has not been detected during the column. Otherwisethe pixel number of the detected location is returned.

// Try upper region first count = 0 for (i=0; i<UPPER_REGION_BOUND; i++){ if (GetPixel(column, i) < THRESHOLD) {       count = 0         //pixel is black   }   else   { count++           // pixel is white if(count > WHITE_ALTERNATIVE ARTCARD)               return i   } } // Trylower region next. Process pixels in reverse count = 0 for(i=MAX_PIXEL_BOUND; i>LOWER_REGION_BOUND; i−−) { if (GetPixel(column, i)< THRESHOLD) { count = 0          // pixel is black } else {count++          // pixel is white if (count > WHITE_ALTERNATIVEARTCARD)               return i   } } //Not in upper bound or in lowerbound. Return failure return 0Calculate Data Block Bounds

At this stage, the alternative Artcard has been detected. Depending onthe rotation of the alternative Artcard, either the top of thealternative Artcard has been detected or the lower part of thealternative Artcard has been detected. The second step of Process 2determines which was detected and sets the data block bounds for Phase 3appropriately.

A look at Phase 3 reveals that it works on data block segment bounds:each data block has a StartPixel and an EndPixel to determine where tolook for targets in order to locate the data block's data region.

If the pixel value is in the upper half of the card, it is possible tosimply use that as the first StartPixel bounds. If the pixel value is inthe lower half of the card, it is possible to move back so that thepixel value is the last segment's EndPixel bounds. We step forwards orbackwards by the alternative Artcard data size, and thus set up eachsegment with appropriate bounds. We are now ready to begin extractingdata from the alternative Artcard.

// Adjust to become first pixel if is lower pixel if (pixel >LOWER_REGION_BOUND) { pixel −= 6 * 1152 if (pixel < 0)     pixel = 0 }for (i=0; i<6; i++) { endPixel = pixel + 1152 segment[i].MaxPixel =MAX_PIXEL_BOUND segment[i].SetBounds(pixel, endPixel) pixel = endPixel }

The MaxPixel value is defined in Process 3, and the SetBounds functionsimply sets StartPixel and EndPixel clipping with respect to 0 andMaxPixel.

Process 3—Extract Bit Data From Pixels

This is the heart of the alternative Artcard Reader algorithm. Thisprocess is concerned with extracting the bit data from the CCD pixeldata. The process essentially creates a bit-image from the pixel data,based on scratch information created by Process 2, and maintained byProcess 3. A high level overview of the process can be seen in FIG. 41.

Rather than simply read an alternative Artcard's pixel column anddetermine what pixels belong to what data block, Process 3 works theother way around. It knows where to look for the pixels of a given datablock. It does this by dividing a logical alternative Artcard into 8segments, each containing 8 data blocks as shown in FIG. 42.

The segments as shown match the logical alternative Artcard. Physically,the alternative Artcard is likely to be rotated by some amount. Thesegments remain locked to the logical alternative Artcard structure, andhence are rotation-independent. A given segment can have one of twostates:

-   -   LookingForTargets: where the exact data block position for this        segment has not yet been determined. Targets are being located        by scanning pixel column data in the bounds indicated by the        segment bounds. Once the data block has been located via the        targets, and bounds set for black & white, the state changes to        ExtractingBitImage.    -   ExtractingBitImage: where the data block has been accurately        located, and bit data is being extracted one dot column at a        time and written to the alternative Artcard bit image. The        following of data block clockmarks gives accurate dot recovery        regardless of rotation, and thus the segment bounds are ignored.        Once the entire data block has been extracted, new segment        bounds are calculated for the next data block based on the        current position. The state changes to LookingForTargets.

The process is complete when all 64 data blocks have been extracted, 8from each region.

Each data block consists of 595 columns of data, each with 48 bytes.Preferably, the 2 orientation columns for the data block are eachextracted at 48 bytes each, giving a total of 28,656 bytes extracted perdata block. For simplicity, it is possible to divide the 2 MB of memoryinto 64×32 k chunks. The nth data block for a given segment is stored atthe location:

StartBuffer+(256 k*n)

Data Structure for Segments

Each of the 8 segments has an associated data structure. The datastructure defining each segment is stored in the scratch data area. Thestructure can be as set out in the following table:

DataName Comment CurrentState Defines the current state of the segment.Can be one of: LookingForTargets ExtractingBitImage Initial value isLookingForTargets Used during LookingForTargets: StartPixel Upper pixelbound of segment. Initially set by Process 2. EndPixel Lower pixel boundof segment. Initially set by Process 2 MaxPixel The maximum pixel numberfor any scanline. It is set to the same value for each segment: 10,866.CurrentColumn Pixel column we're up to while looking for targets.FinalColumn Defines the last pixel column to look in for targets.LocatedTargets Points to a list of located Targets. PossibleTargetsPoints to a set of pointers to Target structures that representcurrently investigated pixel shapes that may be targets AvailableTargetsPoints to a set of pointers to Target structures that are currentlyunused. TargetsFound The number of Targets found so far in this datablock. PossibleTargetCount The number of elements in the PossibleTargetslist AvailabletargetCount The number of elements in the AvailableTargetslist Used during ExtractingBitImage: BitImage The start of the Bit Imagedata area in DRAM where to store the next data block: Segment 1 = X,Segment 2 = X+32k etc Advances by 256k each time the state changes fromExtractingBitImageData to Looking ForTargets CurrentByte Offset withinBitImage where to store next extracted byte CurrentDotColumn Holdscurrent clockmark/dot column number. Set to −8 when transitioning fromstate LookingForTarget to ExtractingBitImage. UpperClock Coordinate(column/pixel) of current upper clockmark/border LowerClock Coordinate(column/pixel) of current lower clockmark/border CurrentDot The centerof the current data dot for the current dot column. Initially set to thecenter of the first (topmost) dot of the data column. DataDelta What toadd (column/pixel) to CurrentDot to advance to the center of the nextdot. BlackMax Pixel value above which a dot is definitely white WhiteMinPixel value below which a dot is definitely black MidRange The pixelvalue that has equal likelihood of coming from black or white. When allsmarts have not determined the dot, this value is used to determine it.Pixels below this value are black, and above it are white.High Level of Process 3

Process 3 simply iterates through each of the segments, performing asingle line of processing depending on the segment's current state. Thepseudocode is straightforward:

blockCount = 0 while (blockCount < 64)   for (i=0; i<8; i++)   {  finishedBlock = segment[i].ProcessState( )   if (finishedBlock)        blockCount++ }

Process 3 must be halted by an external controlling process if it hasnot terminated after a specified amount of time. This will only be thecase if the data cannot be extracted. A simple mechanism is to start acountdown after Process 1 has finished reading the alternative Artcard.If Process 3 has not finished by that time, the data from thealternative Artcard cannot be recovered.

CurrentState=LookingForTargets

Targets are detected by reading columns of pixels, one pixel-column at atime rather than by detecting dots within a given band of pixels(between StartPixel and EndPixel) certain patterns of pixels aredetected. The pixel columns are processed one at a time until either allthe targets are found, or until a specified number of columns have beenprocessed. At that time the targets can be processed and the data arealocated via clockmarks. The state is changed to ExtractingBitImage tosignify that the data is now to be extracted. If enough valid targetsare not located, then the data block is ignored, skipping to a columndefinitely within the missed data block, and then beginning again theprocess of looking for the targets in the next data block. This can beseen in the following pseudocode:

finishedBlock = FALSE if(CurrentColumn < Process1.CurrentScanLine) {ProcessPixelColumn( ) CurrentColumn++ } if ((TargetsFound == 6) ∥(CurrentColumn > LastColumn)) { if (TargetsFound >= 2)        ProcessTargets( )   if (TargetsFound >= 2)   {BuildClockmarkEstimates( ) SetBlackAndWhiteBounds( ) CurrentState =ExtractingBitImage CurrentDotColumn = −8   }   else   { // data blockcannot be recovered. Look for // next instead. Must adjust pixel boundsto // take account of possible 1 degree rotation. finishedBlock = TRUESetBounds(StartPixel−12, EndPixel+12) BitImage += 256KB CurrentByte = 0LastColumn += 1024 TargetsFound = 0   } } return finishedBlockProcessPixelColumn

Each pixel column is processed within the specified bounds (betweenStartPixel and EndPixel) to search for certain patterns of pixels whichwill identify the targets. The structure of a single target (targetnumber 2) is as previously shown in FIG. 23:

From a pixel point of view, a target can be identified by:

-   -   Left black region, which is a number of pixel columns consisting        of large numbers of contiguous black pixels to build up the        first part of the target.    -   Target center, which is a white region in the center of further        black columns    -   Second black region, which is the 2 black dot columns after the        target center    -   Target number, which is a black-surrounded white region that        defines the target number by its length    -   Third black region, which is the 2 black columns after the        target number

An overview of the required process is as shown in FIG. 43.

Since identification only relies on black or white pixels, the pixels1150 from each column are passed through a filter 1151 to detect blackor white, and then run length encoded 1152. The run-lengths are thenpassed to a state machine 1153 that has access to the last 3 run lengthsand the 4th last color. Based on these values, possible targets passthrough each of the identification stages.

The GatherMin&Max process 1155 simply keeps the minimum & maximum pixelvalues encountered during the processing of the segment. These are usedonce the targets have been located to set BlackMax, WhiteMin, andMidRange values.

Each segment keeps a set of target structures in its search for targets.While the target structures themselves don't move around in memory,several segment variables point to lists of pointers to these targetstructures. The three pointer lists are repeated here:

LocatedTargets Points to a set of Target structures that representlocated targets. PossibleTargets Points to a set of pointers to Targetstructures that represent currently investigated pixel shapes that maybe targets. AvailableTargets Points to a set of pointers to Targetstructures that are currently unused.

There are counters associated with each of these list pointers:TargetsFound, PossibleTargetCount, and AvailableTargetCountrespectively.

Before the alternative Artcard is loaded, TargetsFound andPossibleTargetCount are set to 0, and AvailableTargetCount is set to 28(the maximum number of target structures possible to have underinvestigation since the minimum size of a target border is 40 pixels,and the data area is approximately 1152 pixels). An example of thetarget pointer layout is as illustrated in FIG. 44.

As potential new targets are found, they are taken from theAvailableTargets list 1157, the target data structure is updated, andthe pointer to the structure is added to the PossibleTargets list 1158.When a target is completely verified, it is added to the LocatedTargetslist 1159. If a possible target is found not to be a target after all,it is placed back onto the AvailableTargets list 1157. Consequentlythere are always 28 target pointers in circulation at any time, movingbetween the lists.

The Target data structure 1160 can have the following form:

DataName Comment CurrentState The current state of the target searchDetectCount Counts how long a target has been in a given stateStartPixel Where does the target start? All the lines of pixels in thistarget should start within a tolerance of this pixel value. TargetNumberWhich target number is this (according to what was read) Column Bestestimate of the target's center column ordinate Pixel Best estimate ofthe target's center pixel ordinate

The ProcessPixelColumn function within the find targets module 1162(FIG. 43) then, goes through all the run lengths one by one, comparingthe runs against existing possible targets (via StartPixel), or creatingnew possible targets if a potential target is found where none waspreviously known. In all cases, the comparison is only made if S0.coloris white and S1.color is black.

The pseudocode for the ProcessPixelColumn set out hereinafter. When thefirst target is positively identified, the last column to be checked fortargets can be determined as being within a maximum distance from it.For 1° rotation, the maximum distance is 18 pixel columns.

pixel = StartPixel t = 0 target=PossibleTarget[t] while ((pixel <EndPixel) && (TargetsFound < 6)) {  if((S0.Color == white) && (S1.Color== black))  {    do    {     keepTrying = FALSE     if     (     (target != NULL)      &&      (target->AddToTarget(Column, pixel,S1, S2, S3))     )     {      if (target->CurrentState == IsATarget)     {       Remove target from PossibleTargets List       Add target toLocatedTargets List       TargetsFound++       if(TargetsFound == 1)       FinalColumn = Column + MAX_TARGET_DELTA}      }      else if(target->CurrentState == NotATarget)      {       Remove target fromPossibleTargets List       Add target to AvailableTargets List      keepTrying = TRUE      }      else      {       t++    // advanceto next target      }      target = PossibleTarget[t]     }     else    {      tmp = AvailableTargets[0]      if(tmp->AddToTarget(Column,pixel,S1,S2,S3)      {       Remove tmp fromAvailableTargets list       Add tmp to PossibleTargets list      t++   // target t has been shifted right      }     }    } while(keepTrying)   }  pixel += S1.RunLength  Advance S0/S1/S2/S3 }

AddToTarget is a function within the find targets module that determineswhether it is possible or not to add the specific run to the giventarget:

-   -   If the run is within the tolerance of target's starting        position, the run is directly related to the current target, and        can therefore be applied to it.    -   If the run starts before the target, we assume that the existing        target is still ok, but not relevant to the run. The target is        therefore left unchanged, and a return value of FALSE tells the        caller that the run was not applied. The caller can subsequently        check the run to see if it starts a whole new target of its own.    -   If the run starts after the target, we assume the target is no        longer a possible target. The state is changed to be NotATarget,        and a return value of TRUE is returned.

If the run is to be applied to the target, a specific action isperformed based on the current state and set of runs in S1, S2, and S3.The AddToTarget pseudocode is as follows:

MAX_TARGET_DELTA = 1 if (CurrentState != NothingKnown) {   if (pixel >StartPixel)     // run starts after target   {     diff = pixel −StartPixel     if (diff > MAX_TARGET_DELTA)     {       CurrentState =NotATarget       return TRUE     }   }   else   {     diff = StartPixel− pixel     if (diff > MAX_TARGET_DELTA)       return FALSE     } }runType = DetermineRunType(S1, S2, S3) EvaluateState(runType) StartPixel= currentPixel return TRUE

Types of pixel runs are identified in DetermineRunType is as follows:

Types of Pixel Runs Type How identified (S1 is always black)TargetBorder S1 = 40 < RunLength < 50 S2 = white run TargetCenter S1 =15 < RunLength < 26 S2 = white run with [RunLength < 12] S3 = black runwith [15 < RunLength < 26] TargetNumber S2 = white run with [RunLength<= 40]

The EvaluateState procedure takes action depending on the current stateand the run type.

The actions are shown as follows in tabular form:

Type CurrentState of Pixel Run Action NothingKnown TargetBorderDetectCount = 1 CurrentState = LeftOfCenter LeftOfCenter TargetBorderDetectCount++ if (DetectCount > 24)  CurrentState = NotATargetTargetCenter DetectCount = 1 CurrentState = InCenter Column =currentColumn Pixel = currentPixel + S1.RunLength CurrentState =NotATarget InCenter TargetCenter DetectCount++ tmp = currentPixel +S1.RunLength if (tmp < Pixel)  Pixel = tmp if (DetectCount > 13) CurrentState = NotATarget TargetBorder DetectCount = 1 CurrentState =RightOfCenter CurrentState = NotATarget RightOfCenter TargetBorderDetectCount++ if (DetectCount >= 12)  CurrentState = NotATargetTargetNumber DetectCount = 1 CurrentState = InTargetNumber TargetNumber= (S2.RunLength+ 2)/6 CurrentState = NotATarget InTargetNumberTargetNumber tmp = (S2.RunLength+ 2)/6 if (tmp > TargetNumber) TargetNumber = tmp DetectCount++ if (DetectCount >= 12)  CurrentState =NotATarget TargetBorder if (DetectCount >= 3)  CurrentState = IsATargetelse  CurrentState = NotATarget CurrentState = NotATarget IsATarget or —— NotATargetProcessing Targets

The located targets (in the LocatedTargets list) are stored in the orderthey were located. Depending on alternative Artcard rotation thesetargets will be in ascending pixel order or descending pixel order. Inaddition, the target numbers recovered from the targets may be in error.We may have also have recovered a false target. Before the clockmarkestimates can be obtained, the targets need to be processed to ensurethat invalid targets are discarded, and valid targets have targetnumbers fixed if in error (e.g. a damaged target number due to dirt).Two main steps are involved:

-   -   Sort targets into ascending pixel order    -   Locate and fix erroneous target numbers

The first step is simple. The nature of the target retrieval means thatthe data should already be sorted in either ascending pixel ordescending pixel. A simple swap sort ensures that if the 6 targets arealready sorted correctly a maximum of 14 comparisons is made with noswaps. If the data is not sorted, 14 comparisons are made, with 3 swaps.The following pseudocode shows the sorting process:

for (i = 0; i < TargetsFound−1; i++) {   oldTarget = LocatedTargets[i]  bestPixel = oldTarget->Pixel   best = i   j = i+1   while(j<TargetsFound)   {           if (LocatedTargets[j]-> Pixel <bestPixel)             best = j           j++   }   if (best != i) //move only if necessary           LocatedTargets[i] =LocatedTargets[best]           LocatedTargets[best] = oldTarget   } }

Locating and fixing erroneous target numbers is only slightly morecomplex. One by one, each of the N targets found is assumed to becorrect. The other targets are compared to this “correct” target and thenumber of targets that require change should target N be correct iscounted. If the number of changes is 0, then all the targets mustalready be correct Otherwise the target that requires the fewest changesto the others is used as the base for change. A change is registered ifa given target's target number and pixel position do not correlate whencompared to the “correct” target's pixel position and target number. Thechange may mean updating a target's target number, or it may meanelimination of the target. It is possible to assume that ascendingtargets have pixels in ascending order (since they have already beensorted).

kPixelFactor = 1/(55 * 3) bestTarget = 0 bestChanges = TargetsFound + 1for (i=0; i< TotalTargetsFound; i++) {   numberOfChanges = 0; fromPixel= (LocatedTargets[i])->Pixel fromTargetNumber =LocatedTargets[i].TargetNumber for (j=1; j< TotalTargetsFound; j++) {toPixel = LocatedTargets[j]->Pixel deltaPixel = toPixel − fromPixel if(deltaPixel >= 0)      deltaPixel += PIXELS_BETWEEN_TARGET_CENTRES/2  else      deltaPixel −= PIXELS_BETWEEN_TARGET_CENTRES/2 targetNumber=deltaPixel * kPixelFactor targetNumber += fromTargetNumber if   (     (targetNumber < 1)∥(targetNumber > 6)      ∥      (targetNumber !=LocatedTargets[j]-> TargetNumber)     )      numberOfChanges++ }   if(numberOfChanges < bestChanges)   { bestTarget = i bestChanges =numberOfChanges   }   if (bestChanges < 2)      break; }

In most cases this function will terminate with bestChanges=0, whichmeans no changes are required. Otherwise the changes need to be applied.The functionality of applying the changes is identical to counting thechanges (in the pseudocode above) until the comparison withtargetNumber. The change application is:

if ((targetNumber < 1)∥(targetNumber > TARGETS_PER_BLOCK)) {  LocatedTargets[j] = NULL   TargetsFound-- } else {  LocatedTargets[j]-> TargetNumber = targetNumber }

At the end of the change loop, the LocatedTargets list needs to becompacted and all NULL targets removed.

At the end of this procedure, there may be fewer targets. Whatevertargets remain may now be used (at least 2 targets are required) tolocate the clockmarks and the data region.

Building Clockmark Estimates From Targets

As shown previously in FIG. 24, the upper region's first clockmark dot1126 is 55 dots away from the center of the first target 1124 (which isthe same as the distance between target centers). The center of theclockmark dots is a further 1 dot away, and the black border line 1123is a further 4 dots away from the first clockmark dot The lower region'sfirst clockmark dot is exactly 7 targets-distance away (7×55 dots) fromthe upper region's first clockmark dot 1126.

It cannot be assumed that Targets 1 and 6 have been located, so it isnecessary to use the upper-most and lower-most targets, and use thetarget numbers to determine which targets are being used. It isnecessary at least 2 targets at this point. In addition, the targetcenters are only estimates of the actual target centers. It is to locatethe target center more accurately. The center of a target is white,surrounded by black. We therefore want to find the local maximum in bothpixel & column dimensions. This involves reconstructing the continuousimage since the maximum is unlikely to be aligned exactly on an integerboundary (our estimate).

Before the continuous image can be constructed around the target'scenter, it is necessary to create a better estimate of the 2 targetcenters. The existing target centers actually are the top leftcoordinate of the bounding box of the target center. It is a simpleprocess to go through each of the pixels for the area defining thecenter of the target, and find the pixel with the highest value. Theremay be more than one pixel with the same maximum pixel value, but theestimate of the center value only requires one pixel.

The pseudocode is straightforward, and is performed for each of the 2targets:

CENTER_WIDTH = CENTER_HEIGHT = 12 maxPixel = 0x00 for (i=0;i<CENTER_WIDTH; i++)   for (j=0; j<CENTER_HEIGHT; j++)   { p =GetPixel(column+i, pixel+j) if (p > maxPixel) { maxPixel = pcenterColumn = column + i centerPixel = pixel + j         }   }Target.Column = centerColumn Target.Pixel = centerPixel

At the end of this process the target center coordinates point to thewhitest pixel of the target, which should be within one pixel of theactual center. The process of building a more accurate position for thetarget center involves reconstructing the continuous signal for 7scanline slices of the target, 3 to either side of the estimated targetcenter. The 7 maximum values found (one for each of these pixeldimension slices) are then used to reconstruct a continuous signal inthe column dimension and thus to locate the maximum value in thatdimension.

// Given estimates column and pixel, determine a // betterColumn andbetterPixel as the center of // the target for (y=0; y<7; y++) {   for(x=0; x<7; x++)     samples[x] = GetPixel(column−3+y, pixel−3+x)  FindMax(samples, pos, maxVal)   reSamples[y] = maxVal   if (y == 3)    betterPixel = pos + pixel } FindMax(reSamples, pos, maxVal)betterColumn = pos + column

FindMax is a function that reconstructs the original 1 dimensionalsignal based sample points and returns the position of the maximum aswell as the maximum value found. The method of signalreconstruction/resampling used is the Lanczos3 windowed sinc function asshown in FIG. 45.

The Lanczos3 windowed sinc function takes 7 (pixel) samples from thedimension being reconstructed, centered around the estimated position X,i.e. at X−3, X−2, X−1, X, X+1, X+2, X+3. We reconstruct points from X−1to X+1, each at an interval of 0.1, and determine which point is themaximum. The position that is the maximum value becomes the new center.Due to the nature of the kernel, only 6 entries are required in theconvolution kernel for points between X and X+1. We use 6 points for X−1to X, and 6 points for X to X+1, requiring 7 points overall in order toget pixel values from X−1 to X+1 since some of the pixels required arethe same.

Given accurate estimates for the upper-most target from and lower-mosttarget to, it is possible to calculate the position of the firstclockmark dot for the upper and lower regions as follows:

TARGETS_PER_BLOCK = 6 numTargetsDiff = to.TargetNum − from.TargetNumdeltaPixel = (to.Pixel − from.Pixel) / numTargetsDiff deltaColumn =(to.Column − from.Column) / numTargetsDiff UpperClock.pixel = from.Pixel− (from.TargetNum*deltaPixel) UpperClock.column =from.Column−(from.TargetNum*deltaColumn) // Given the first dot of theupper clockmark, the // first dot of the lower clockmark isstraightforward. LowerClock.pixel        =       UpperClock.pixel   +((TARGETS_PER_BLOCK+1) * deltaPixel) LowerClock.column      =       UpperClock.column   + ((TARGETS_PER_BLOCK+1) *deltaColumn)

This gets us to the first clockmark dot. It is necessary move the columnposition a further 1 dot away from the data area to reach the center ofthe clockmark. It is necessary to also move the pixel position a further4 dots away to reach the center of the border line. The pseudocodevalues for deltaColumn and deltaPixel are based on a 55 dot distance(the distance between targets), so these deltas must be scaled by 1/55and 4/55 respectively before being applied to the clockmark coordinates.This is represented as:

kDeltaDotFactor = 1/DOTS_BETWEEN_TARGET_CENTRES deltaColumn *=kDeltaDotFactor deltaPixel *= 4 * kDeltaDotFactor UpperClock.pixel −=deltaPixel UpperClock.column −= deltaColumn LowerClock.pixel +=deltaPixel LowerClock.column += deltaColumn

UpperClock and LowerClock are now valid clockmark estimates for thefirst clockmarks directly in line with the centers of the targets.

Setting Black and White Pixel/Dot Ranges

Before the data can be extracted from the data area, the pixel rangesfor black and white dots needs to be ascertained. The minimum andmaximum pixels encountered during the search for targets were stored inWhiteMin and BlackMax respectively, but these do not represent validvalues for these variables with respect to data extraction. They aremerely used for storage convenience. The following pseudocode shows themethod of obtaining good values for WhiteMin and BlackMax based on themin & max pixels encountered:

MinPixel = WhiteMin MaxPixel = BlackMax MidRange = (MinPixel + MaxPixel)/ 2 WhiteMin = MaxPixel − 105 BlackMax = MinPixel + 84 CurrentState =ExtractingBitImage

The ExtractingBitImage state is one where the data block has alreadybeen accurately located via the targets, and bit data is currently beingextracted one dot column at a time and written to the alternativeArtcard bit image. The following of data block clockmarks/borders givesaccurate dot recovery regardless of rotation, and thus the segmentbounds are ignored. Once the entire data block has been extracted (597columns of 48 bytes each; 595 columns of data+2 orientation columns),new segment bounds are calculated for the next data block based on thecurrent position The state is changed to LookingForTargets.

Processing a given dot column involves two tasks:

-   -   The first task is to locate the specific dot column of data via        the clockmarks.    -   The second task is to run down the dot column gathering the bit        values, one bit per dot.

These two tasks can only be undertaken if the data for the column hasbeen read off the alternative Artcard and transferred to DRAM. This canbe determined by checking what scanline Process 1 is up to, andcomparing it to the clockmark columns. If the dot data is in DRAM we canupdate the clockmarks and then extract the data from the column beforeadvancing the clockmarks to the estimated value for the next dot column.The process overview is given in the following pseudocode, with specificfunctions explained hereinafter.

finishedBlock = FALSE if((UpperClock.column < Process1.CurrentScanLine)  &&   (LowerClock.column < Process1.CurrentScanLine)) {  DetermineAccurateClockMarks( )   DetermineDataInfo( )   if(CurrentDotColumn >= 0)       ExtractDataFromColumn( )  AdvanceClockMarks( )   if (CurrentDotColumn == FINAL_COLUMN)   {finishedBlock = TRUE currentState = LookingForTargetsSetBounds(UpperClock.pixel, LowerClock.pixel) BitImage += 256KBCurrentByte = 0 TargetsFound = 0   } } return finishedBlockLocating the Dot Column

A given dot column needs to be located before the dots can be read andthe data extracted. This is accomplished by following theclockmarks/borderline along the upper and lower boundaries of the datablock. A software equivalent of a phase-locked-loop is used to ensurethat even if the clockmarks have been damaged, good estimations ofclockmark positions will be made. FIG. 46 illustrates an example datablock's top left which corner reveals that there are clockmarks 3 dotshigh 1166 extending out to the target area, a white row, and then ablack border line.

Initially, an estimation of the center of the first black clockmarkposition is provided (based on the target positions). We use the blackborder 1168 to achieve an accurate vertical position (pixel), and theclockmark eg. 1166 to get an accurate horizontal position (column).These are reflected in the UpperClock and LowerClock positions.

The clockmark estimate is taken and by looking at the pixel data in itsvicinity, the continuous signal is reconstructed and the exact center isdetermined. Since we have broken out the two dimensions into a clockmarkand border, this is a simple one-dimensional process that needs to beperformed twice. However, this is only done every second dot column,when there is a black clockmark to register against. For the whiteclockmarks we simply use the estimate and leave it at that.Alternatively, we could update the pixel coordinate based on the bordereach dot column (since it is always present). In practice it issufficient to update both ordinates every other column (with the blackclockmarks) since the resolution being worked at is so fine. The processtherefore becomes:

// Turn the estimates of the clockmarks into accurate // positions onlywhen there is a black clockmark // (ie every 2nd dot column, startingfrom −8) if (Bit0(CurrentDotColumn) == 0)    // even column {  DetermineAccurateUpperDotCenter( )   DetermineAccurateLowerDotCenter() }

If there is a deviation by more than a given tolerance(MAX_CLOCKMARK_DEVIATION), the found signal is ignored and onlydeviation from the estimate by the maximum tolerance is allowed. In thisrespect the functionality is similar to that of a phase-locked loop.Thus DetermineAccurateUpperDotCenter is implemented via the followingpseudocode:

// Use the estimated pixel position of // the border to determine whereto look for // a more accurate clockmark center. The clockmark // is 3dots high so even if the estimated position // of the border is wrong,it won't affect the // fixing of the clockmark position.MAX_CLOCKMARK_DEVIATION = 0.5 diff      =     GetAccurateColumn(UpperClock.column,        UpperClock.pixel+(3*PIXELS_PER_DOT)) diff −= UpperClock.columnif (diff > MAX_CLOCKMARK_DEVIATION)   diff = MAX_CLOCKMARK_DEVIATIONelse if (diff < −MAX_CLOCKMARK_DEVIATION)   diff =−MAX_CLOCKMARK_DEVIATION UpperClock.column += diff // Use the newlyobtained clockmark center to // determine a more accurate borderposition. diff = GetAccuratePixel(UpperClock.column, UpperClock.pixel)diff −= UpperClock.pixel if (diff > MAX_CLOCKMARK_DEVIATION)   diff =MAX_CLOCKMARK_DEVIATION else if (diff < −MAX_CLOCKMARK_DEVIATION)   diff= −MAX_CLOCKMARK_DEVIATION UpperClock.pixel += diff

DetermineAccurateLowerDotCenter is the same, except that the directionfrom the border to the clockmark is in the negative direction (−3 dotsrather than +3 dots).

GetAccuratePixel and GetAccurateColumn are functions that determine anaccurate dot center given a coordinate, but only from the perspective ofa single dimension. Determining accurate dot centers is a process ofsignal reconstruction and then finding the location where the minimumsignal value is found (this is different to locating a target center,which is locating the maximum value of the signal since the targetcenter is white, not black). The method chosen for signalreconstruction/resampling for this application is the Lanczos3 windowedsinc function as previously discussed with reference to FIG. 45.

It may be that the clockmark or border has been damaged in someway—perhaps it has been scratched. If the new center value retrieved bythe resampling differs from the estimate by more than a toleranceamount, the center value is only moved by the maximum tolerance. If itis an invalid position, it should be close enough to use for dataretrieval, and future clockmarks will resynchronize the position.

Determining the Center of the First Data Dot and the Deltas toSubsequent Dots

Once an accurate UpperClock and LowerClock position has been determined,it is possible to calculate the center of the first data dot(CurrentDot), and the delta amounts to be added to that center positionin order to advance to subsequent dots in the column (DataDelta).

The first thing to do is calculate the deltas for the dot column. Thisis achieved simply by subtracting the UpperClock from the LowerClock,and then dividing by the number of dots between the two points. It ispossible to actually multiply by the inverse of the number of dots sinceit is constant for an alternative Artcard, and multiplying is faster. Itis possible to use different constants for obtaining the deltas in pixeland column dimensions. The delta in pixels is the distance between thetwo borders, while the delta in columns is between the centers of thetwo clockmarks. Thus the function DetermineDatainfo is two parts. Thefirst is given by the pseudocode:

kDeltaColumnFactor = 1 / (DOTS_PER_DATA_COLUMN + 2 + 2 − 1)kDeltaPixelFactor = 1 / (DOTS_PER_DATA_COLUMN + 5 + 5 − 1) delta =LowerClock.column − UpperClock.column DataDelta.column = delta *kDeltaColumnFactor delta = LowerClock.pixel − UpperClock.pixelDataDelta.pixel = delta * kDeltaPixelFactor

It is now possible to determine the center of the first data dot of thecolumn. There is a distance of 2 dots from the center of the clockmarkto the center of the first data dot, and 5 dots from the center of theborder to the center of the first data dot. Thus the second part of thefunction is given by the pseudocode:

CurrentDot.column = UpperClock.column + (2*DataDelta.column)CurrentDot.pixel = UpperClock.pixel + (5*DataDelta.pixel)Running Down a Dot Column

Since the dot column has been located from the phase-locked looptracking the clockmarks, all that remains is to sample the dot column atthe center of each dot down that column. The variable CurrentDot pointsis determined to the center of the first dot of the current column. Wecan get to the next dot of the column by simply adding DataDelta (2additions: 1 for the column ordinate, the other for the pixel ordinate).A sample of the dot at the given coordinate (bi-linear interpolation) istaken, and a pixel value representing the center of the dot isdetermined. The pixel value is then used to determine the bit value forthat dot. However it is possible to use the pixel value in context withthe center value for the two surrounding dots on the same dot line tomake a better bit judgement.

We can be assured that all the pixels for the dots in the dot columnbeing extracted are currently loaded in DRAM, for if the two ends of theline (clockmarks) are in DRAM, then the dots between those twoclockmarks must also be in DRAM. Additionally, the data block height isshort enough (only 384 dots high) to ensure that simple deltas areenough to traverse the length of the line. One of the reasons the cardis divided into 8 data blocks high is that we cannot make the same rigidguarantee across the entire height of the card that we can about asingle data block.

The high level process of extracting a single line of data (48 bytes)can be seen in the following pseudocode. The dataBuffer pointerincrements as each byte is stored, ensuring that consecutive bytes andcolumns of data are stored consecutively.

bitCount = 8 curr = 0x00        // definitely black next =GetPixel(CurrentDot) for (i=0; i < DOTS_PER_DATA_COLUMN; i++) {  CurrentDot += DataDelta   prev = curr   curr = next   next =GetPixel(CurrentDot)   bit = DetermineCenterDot(prev, curr, next)   byte= (byte << 1) | bit   bitCount−−   if (bitCount == 0)   {     *(BitImage| CurrentByte) = byte     CurrentByte++     bitCount = 8 } }

The GetPixel function takes a dot coordinate (fixed point) and samples 4CCD pixels to arrive at a center pixel value via bilinear interpolation.

The DetermineCenterDot function takes the pixel values representing thedot centers to either side of the dot whose bit value is beingdetermined, and attempts to intelligently guess the value of that centerdot's bit value. From the generalized blurring curve of FIG. 33 thereare three common cases to consider:

-   -   The dot's center pixel value is lower than WhiteMin, and is        therefore definitely a black dot. The bit value is therefore        definitely 1.    -   The dot's center pixel value is higher than BlackMax, and is        therefore definitely a white dot. The bit value is therefore        definitely 0.    -   The dot's center pixel value is somewhere between BlackMax and        WhiteMin. The dot may be black, and it may be white. The value        for the bit is therefore in question. A number of schemes can be        devised to make a reasonable guess as to the value of the bit.        These schemes must balance complexity against accuracy, and also        take into account the fact that in some cases, there is no        guaranteed solution. In those cases where we make a wrong bit        decision, the bit's Reed-Solomon symbol will be in error, and        must be corrected by the Reed-Solomon decoding stage in Phase 2.

The scheme used to determine a dot's value if the pixel value is betweenBlackMax and WhiteMin is not too complex, but gives good results. Ituses the pixel values of the dot centers to the left and right of thedot in question, using their values to help determine a more likelyvalue for the center dot:

-   -   If the two dots to either side are on the white side of MidRange        (an average dot value), then we can guess that if the center dot        were white, it would likely be a “definite” white. The fact that        it is in the not-sure region would indicate that the dot was        black, and had been affected by the surrounding white dots to        make the value less sure. The dot value is therefore assumed to        be black, and hence the bit value is 1.    -   If the two dots to either side are on the black side of        MidRange, then we can guess that if the center dot were black,        it would likely be a “definite” black. The fact that it is in        the not-sure region would indicate that the dot was white, and        had been affected by the surrounding black dots to make the        value less sure. The dot value is therefore assumed to be white,        and hence the bit value is 0.    -   If one dot is on the black side of MidRange, and the other dot        is on the white side of MidRange, we simply use the center dot        value to decide. If the center dot is on the black side of        MidRange, we choose black (bit value 1). Otherwise we choose        white (bit value 0).

The logic is represented by the following:

if (pixel < WhiteMin)      // definitely black      bit = 0x01 else if(pixel > BlackMax)      // definitely white      bit = 0x00 else if((prev > MidRange) && (next> MidRange)) //prob black      bit = 0x01else if ((prev < MidRange) && (next < MidRange)) //prob white      bit =0x00 else if (pixel < MidRange)      bit = 0x01 else      bit = 0x00

From this one can see that using surrounding pixel values can give agood indication of the value of the center dot's state. The schemedescribed here only uses the dots from the same row, but using a singledot line history (the previous dot line) would also be straightforwardas would be alternative arrangements.

Updating Clockmarks for the Next Column

Once the center of the first data dot for the column has beendetermined, the clockmark values are no longer needed. They areconveniently updated in readiness for the next column after the data hasbeen retrieved for the column. Since the clockmark direction isperpendicular to the traversal of dots down the dot column, it ispossible to use the pixel delta to update the column, and subtract thecolumn delta to update the pixel for both clocks:

UpperClock.column += DataDelta.pixel LowerClock.column +=DataDelta.pixel UpperClock.pixel −= DataDelta.column LowerClock.pixel −=DataDelta.column

These are now the estimates for the next dot column.

Timing

The timing requirement will be met as long as DRAM utilization does notexceed 100%, and the addition of parallel algorithm timing multiplied bythe algorithm DRAM utilization does not exceed 100%. DRAM utilization isspecified relative to Process1, which writes each pixel once in aconsecutive manner, consuming 9% of the DRAM bandwidth.

The timing as described in this section, shows that the DRAM is easilyable to cope with the demands of the alternative Artcard Readeralgorithm. The timing bottleneck will therefore be the implementation ofthe algorithm in terms of logic speed, not DRAM access. The algorithmshave been designed however, with simple architectures in mind, requiringa minimum number of logical operations for every memory cycle. From thispoint of view, as long as the implementation state machine or equivalentCPU/DSP architecture is able to perform as described in the followingsub-sections, the target speed will be met

Locating the Targets

Targets are located by reading pixels within the bounds of a pixelcolumn. Each pixel is read once at most. Assuming a run-length encoderthat operates fast enough, the bounds on the location of targets ismemory access. The accesses will therefore be no worse than the timingfor Process 1, which means a 9% utilization of the DRAM bandwidth.

The total utilization of DRAM during target location (includingProcess1) is therefore 18%, meaning that the target locator will alwaysbe catching up to the alternative Artcard image sensor pixel reader.

Processing the Targets

The timing for sorting and checking the target numbers is trivial. Thefinding of better estimates for each of the two target centers involves12 sets of 12 pixel reads, taking a total of 144 reads. However thefixing of accurate target centers is not trivial, requiring 2 sets ofevaluations. Adjusting each target center requires 8 sets of 20different 6-entry convolution kernels. Thus this totals 8×20×6multiply-accumulates=960. In addition, there are 7 sets of 7 pixels tobe retrieved, requiring 49 memory accesses. The total number per targetis therefore 144+960+49=1153, which is approximately the same number ofpixels in a column of pixels (1152). Thus each target evaluationconsumes the time taken by otherwise processing a row of pixels. For twotargets we effectively consume the time for 2 columns of pixels.

A target is positively identified on the first pixel column after thetarget number. Since there are 2 dot columns before the orientationcolumn, there are 6 pixel columns. The Target Location processeffectively uses up the first of the pixel columns, but the remaining 5pixel columns are not processed at all. Therefore the data area can belocated in ⅖ of the time available without impinging on any otherprocess time.

The remaining ⅗ of the time available is ample for the trivial task ofassigning the ranges for black and white pixels, a task that may take acouple of machine cycles at most.

Extracting Data

There are two parts to consider in terms of timing:

-   -   Getting accurate clockmarks and border values    -   Extracting dot values

Clockmarks and border values are only gathered every second dot column.However each time a clockmark estimate is updated to become moreaccurate, 20 different 6-entry convolution kernels must be evaluated. Onaverage there are 2 of these per dot column (there are 4 every 2dot-columns). Updating the pixel ordinate based on the border onlyrequires 7 pixels from the same pixel scanline. Updating the columnordinate however, requires 7 pixels from different columns, hencedifferent scanlines. Assuming worst case scenario of a cache miss foreach scanline entry and 2 cache misses for the pixels in the samescanline, this totals 8 cache misses.

Extracting the dot information involves only 4 pixel reads per dot(rather than the average 9 that define the dot). Considering the dataarea of 1152 pixels (384 dots), at best this will save 72 cache reads byonly reading 4 pixel dots instead of 9. The worst case is a rotation of1° which is a single pixel translation every 57 pixels, which gives onlyslightly worse savings.

It can then be safely said that, at worst, we will be reading fewercache lines less than that consumed by the pixels in the data area. Theaccesses will therefore be no worse than the timing for Process 1, whichimplies a 9% utilization of the DRAM bandwidth.

The total utilization of DRAM during data extraction (includingProcess1) is therefore 18%, meaning that the data extractor will alwaysbe catching up to the alternative Artcard image sensor pixel reader.This has implications for the Process Targets process in that theprocessing of targets can be performed by a relatively inefficientmethod if necessary, yet still catch up quickly during the extractingdata process.

Phase 2—Decode Bit Image

Phase 2 is the non-real-time phase of alternative Artcard data recoveryalgorithm. At the start of Phase 2 a bit image has been extracted fromthe alternative Artcard. It represents the bits read from the dataregions of the alternative Artcard. Some of the bits will be in error,and perhaps the entire data is rotated 180° because the alternativeArtcard was rotated when inserted. Phase 2 is concerned with reliablyextracting the original data from this encoded bit image. There arebasically 3 steps to be carried out as illustrated in FIG. 48:

-   -   Reorganize the bit image, reversing it if the alternative        Artcard was inserted backwards    -   Unscramble the encoded data    -   Reed-Solomon decode the data from the bit image

Each of the 3 steps is defined as a separate process, and performedconsecutively, since the output of one is required as the input to thenext. It is straightforward to combine the first two steps into a singleprocess, but for the purposes of clarity, they are treated separatelyhere.

From a data/process perspective, Phase 2 has the structure asillustrated in FIG. 49.

The timing of Processes 1 and 2 are likely to be negligible, consumingless than 1/1000^(th) of a second between them. Process 3 (Reed Solomondecode) consumes approximately 0.32 seconds, making this the total timerequired for Phase 2.

Reorganize the Bit Image, Reversing it if Necessary

The bit map in DRAM now represents the retrieved data from thealternative Artcard. However the bit image is not contiguous. It isbroken into 64 32 k chunks, one chunk for each data block. Each 32 kchunk contains only 28,656 useful bytes:

-   48 bytes from the leftmost Orientation Column-   28560 bytes from the data region proper-   48 bytes from the rightmost Orientation Column-   4112 unused bytes

The 2 MB buffer used for pixel data (stored by Process 1 of Phase 1) canbe used to hold the reorganized bit image, since pixel data is notrequired during Phase 2. At the end of the reorganization, a correctlyoriented contiguous bit image will be in the 2 MB pixel buffer, readyfor Reed-Solomon decoding.

If the card is correctly oriented, the leftmost Orientation Column willbe white and the rightmost Orientation Column will be black. If the cardhas been rotated 180°, then the leftmost Orientation Column will beblack and the rightmost Orientation Column will be white.

A simple method of determining whether the card is correctly oriented ornot, is to go through each data block, checking the first and last 48bytes of data until a block is found with an overwhelming ratio of blackto white bits. The following pseudocode demonstrates this, returningTRUE if the card is correctly oriented, and FALSE if it is not:

totalCountL = 0 totalCountR = 0 for (i=0; i<64; i++) { blackCountL = 0blackCountR = 0 currBuff = dataBuffer for (j=0; j<48; j++) { blackCountL+= CountBits(*currBuff) currBuff++   } currBuff += 28560 for (j=0; j<48;j++) { blackCountR += CountBits(*currBuff) currBuff++   } dataBuffer +=32k if (blackCountR > (blackCountL * 4))      return TRUE   if(blackCountL > (blackCountR * 4))      return FALSE   totalCountL +=blackCountL   totalCountR += blackCountR } return (totalCountR >totalCountL)

The data must now be reorganized, based on whether the card was orientedcorrectly or not. The simplest case is that the card is correctlyoriented. In this case the data only needs to be moved around a littleto remove the orientation columns and to make the entire datacontiguous. This is achieved very simply in situ, as described by thefollowing pseudocode:

DATA_BYTES_PER_DATA_BLOCK = 28560 to = dataBuffer from = dataBuffer +48)    // left orientation column for (i=0; i<64; i++) { BlockMove(from,to, DATA_BYTES_PER_DATA_BLOCK) from += 32k to +=DATA_BYTES_PER_DATA_BLOCK }

The other case is that the data actually needs to be reversed. Thealgorithm to reverse the data is quite simple, but for simplicity,requires a 256-byte table Reverse where the value of Reverse[N] is abit-reversed N.

DATA_BYTES_PER_DATA_BLOCK = 28560 to = outBuffer for (i=0; i<64; i++) {from = dataBuffer + (i * 32k) from += 48          //skip orientationcolumn from += DATA_BYTES_PER_DATA_BLOCK − 1 // end of block for (j=0; j< DATA_BYTES_PER_DATA_BLOCK; j++) { *to++ = Reverse[*from] from−−   } }

The timing for either process is negligible, consuming less than1/1000^(th) of a second:

-   -   2 MB contiguous reads (2048/16×12 ns=1,536 ns)    -   2 MB effectively contiguous byte writes (2048/16×12 ns=1,536 ns)        Unscramble the Encoded Image

The bit image is now 1,827,840 contiguous, correctly oriented, butscrambled bytes. The bytes must be unscrambled to create the 7,168Reed-Solomon blocks, each 255 bytes long. The unscrambling process isquite straightforward, but requires a separate output buffer since theunscrambling cannot be performed in situ. FIG. 49 illustrates theunscrambling process conducted memory

The following pseudocode defines how to perform the unscramblingprocess:

groupSize = 255 numBytes = 1827840; inBuffer = scrambledBuffer;outBuffer = unscrambledBuffer; for (i=0; i<groupSize; i++)   for (j=i;j<numBytes; j+=groupSize)       outBuffer[j] = *inBuffer++

The timing for this process is negligible, consuming less than1/1000^(th) of a second:

-   -   2 MB contiguous reads (2048/16×12 ns=1,536 ns)    -   2 MB non-contiguous byte writes (2048×12 ns=24,576 ns)

At the end of this process the unscrambled data is ready forReed-Solomon decoding.

Reed Solomon Decode

The final part of reading an alternative Artcard is the Reed-Solomondecode process, where approximately 2 MB of unscrambled data is decodedinto approximately 1 MB of valid alternative Artcard data.

The algorithm performs the decoding one Reed-Solomon block at a time,and can (if desired) be performed in situ, since the encoded block islarger than the decoded block, and the redundancy bytes are stored afterthe data bytes.

The first 2 Reed-Solomon blocks are control blocks, containinginformation about the size of the data to be extracted from the bitimage. This meta-information must be decoded first, and the resultantinformation used to decode the data proper. The decoding of the dataproper is simply a case of decoding the data blocks one at a time.Duplicate data blocks can be used if a particular block fails to decode.

The highest level of the Reed-Solomon decode is set out in pseudocode:

// Constants for Reed Solomon decode sourceBlockLength = 255;destBlockLength = 127; numControlBlocks = 2; // Decode the controlinformation if(! GetControlData(source, destBlocks, lastBlock))   returnerror destBytes = ((destBlocks−1) * destBlockLength) + lastBlockoffsetToNextDuplicate = destBlocks * sourceBlockLength // Skip thecontrol blocks and position at data source += numControlBlocks *sourceBlockLength // Decode each of the data blocks, trying //duplicates as necessary blocksInError = 0; for (i=0; i<destBlocks; i++){   found = DecodeBlock(source, dest); if(! found)   { duplicate =source + offsetToNextDuplicate while ((! found) &&(duplicate<sourceEnd))       { found = DecodeBlock(duplicate, dest)duplicate += offsetToNextDuplicate       }   }   if(! found)      blocksInError++   source += sourceBlockLength   dest +=destBlockLength } return destBytes and blocksInError

DecodeBlock is a standard Reed Solomon block decoder using m=8 and t=64.

The GetControlData function is straightforward as long as there are nodecoding errors. The function simply calls DecodeBlock to decode onecontrol block at a time until successful. The control parameters canthen be extracted from the first 3 bytes of the decoded data (destblocksis stored in the bytes 0 and 1, and lastblock is stored in byte 2). Ifthere are decoding errors the function must traverse the 32 sets of 3bytes and decide which is the most likely set value to be correct Onesimple method is to find 2 consecutive equal copies of the 3 bytes, andto declare those values the correct ones. An alternative method is tocount occurrences of the different sets of 3 bytes, and announce themost common occurrence to be the correct one.

The time taken to Reed-Solomon decode depends on the implementation.While it is possible to use a dedicated core to perform the Reed-Solomondecoding process (such as LSI Logic's L64712), it is preferable toselect a CPU/DSP combination that can be more generally used throughoutthe embedded system (usually to do something with the decoded data)depending on the application. Of course decoding time must be fastenough with the CPU/DSP combination.

The L64712 has a throughput of 50 Mbits per second (around 6.25 MB persecond), so the time is bound by the speed of the Reed-Solomon decoderrather than the maximum 2 MB read and 1 MB write memory access time. Thetime taken in the worst case (all 2 MB requires decoding) is thus 2/6.25s=approximately 0.32 seconds. Of course, many further refinements arepossible including the following:

The blurrier the reading environment, the more a given dot is influencedby the surrounding dots. The current reading algorithm of the preferredembodiment has the ability to use the surrounding dots in the samecolumn in order to make a better decision about a dot's value. Since theprevious column's dots have already been decoded, a previous column dothistory could be useful in determining the value of those dots whosepixel values are in the not-sure range.

A different possibility with regard to the initial stage is to remove itentirely, make the initial bounds of the data blocks larger thannecessary and place greater intelligence into the ProcessingTargetsfunctions. This may reduce overall complexity. Care must be taken tomaintain data block independence.

Further the control block mechanism can be made more robust:

-   -   The control block could be the first and last blocks rather than        make them contiguous (as is the case now). This may give greater        protection against certain pathological damage scenarios.    -   The second refinement is to place an additional level of        redundancy/error detection into the control block structure to        be used if the Reed-Solomon decode step fails. Something as        simple as parity might improve the likelihood of control        information if the Reed-Solomon stage fails.        Data Card Reader

FIG. 51, there is illustrated one form of card reader 500 which allowsfor the insertion of Artcards 9 for reading. FIG. 50 shows an explodedperspective of the reader of FIG. 51. Cardreader is interconnected to acomputer system and includes a CCD reading mechanism 35. The cardreaderincludes pinch rollers 506, 507 for pinching an inserted Artcard 9. Oneof the roller e.g. 506 is driven by an Artcard motor 37 for theadvancement of the card 9 between the two rollers 506 and 507 at auniformed speed. The Artcard 9 is passed over a series of LED lights 512which are encased within a clear plastic mould 514 having a semicircular cross section. The cross section focuses the light from theLEDs eg 512 onto the surface of the card 9 as it passes by the LEDs 512.From the surface it is reflected to a high resolution linear CCD 34which is constructed to a resolution of approximately 480 dpi. Thesurface of the Artcard 9 is encoded to the level of approximately 1600dpi hence, the linear CCD 34 supersamples the Artcard surface with anapproximately three times multiplier. The Artcard 9 is further driven ata speed such that the linear CCD 34 is able to supersample in thedirection of Artcard movement at a rate of approximately 4800 readingsper inch. The scanned Artcard CCD data is forwarded from the Artcardreader to ACP 31 for processing. A sensor 49, which can comprise a lightsensor acts to detect of the presence of the card 13.

The CCD reader includes a bottom substrate 516, a top substrate 514which comprises a transparent molded plastic. In between the twosubstrates is inserted the linear CCD array 34 which comprises a thinlong linear CCD array constructed by means of semi-conductormanufacturing processes.

Turning to FIG. 52, there is illustrated a side perspective view, partlyin section, of an example construction of the CCD reader unit. Theseries of LEDs eg. 512 are operated to emit light when a card 9 ispassing across the surface of the CCD reader 34. The emitted light istransmitted through a portion of the top substrate 523. The substrateincludes a portion eg. 529 having a curved circumference so as to focuslight emitted from LED 512 to a point eg. 532 on the surface of the card9. The focused light is reflected from the point 532 towards the CCDarray 34. A series of microlenses eg. 534, shown in exaggerated form,are formed on the surface of the top substrate 523. The microlenses 523act to focus light received across the surface to the focused down to apoint 536 which corresponds to point on the surface of the CCD reader 34for sensing of light falling on the light sensing portion of the CCDarray 34.

A number of refinements of the above arrangement are possible. Forexample, the sensing devices on the linear CCD 34 may be staggered. Thecorresponding microlenses 34 can also be correspondingly formed as tofocus light into a staggered series of spots so as to correspond to thestaggered CCD sensors.

To assist reading, the data surface area of the Artcard 9 is modulatedwith a checkerboard pattern as previously discussed with reference toFIG. 5. Other forms of high frequency modulation may be possiblehowever.

It will be evident that an Artcard printer can be provided as for theprinting out of data on storage Artcard. Hence, the Artcard system canbe utilized as a general form of information distribution outside of theArtcam device. An Artcard printer can prints out Artcards on highquality print surfaces and multiple Artcards can be printed on samesheets and later separated. On a second surface of the Artcard 9 can beprinted information relating to the files etc. stored on the Artcard 9for subsequent storage.

Hence, the Artcard system allows for a simplified form of storage whichis suitable for use in place of other forms of storage such as CD ROMs,magnetic disks etc. The Artcards 9 can also be mass produced and therebyproduced in a substantially inexpensive form for redistribution.

Print Rolls

Turning to FIG. 54, there is illustrated the print roll 42 andprint-head portions of the Artcam. The paper/film 611 is fed in acontinuous “web-like” process to a printing mechanism 15 which includesfurther pinch rollers 616–619 and a print head 44.

The pinch roller 613 is connected to a drive mechanism (not shown) andupon rotation of the print roller 613, “paper” in the form of film 611is forced through the printing mechanism 615 and out of the pictureoutput slot 6. A rotary guillotine mechanism (not shown) is utilised tocut the roll of paper 611 at required photo sizes.

It is therefore evident that the printer roll 42 is responsible forsupplying “paper” 611 to the print mechanism 615 for printing ofphotographically imaged pictures.

In FIG. 55, there is shown an exploded perspective of the print roll 42.The printer roll 42 includes output printer paper 611 which is outputunder the operation of pinching rollers 612, 613.

Referring now to FIG. 56, there is illustrated a more filly explodedperspective view, of the print roll 42 of FIG. 55 without the “paper”film roll. The print roll 42 includes three main parts comprising inkreservoir section 620, paper roll sections 622, 623 and outer casingsections 626, 627.

Turning first to the ink reservoir section 620, which includes the inkreservoir or ink supply sections 633. The ink for printing is containedwithin three bladder type containers 630–632. The printer roll 42 isassumed to provide full color output inks. Hence, a first ink reservoiror bladder container 630 contains cyan colored ink. A second reservoir631 contains magenta colored ink and a third reservoir 632 containsyellow ink. Each of the reservoirs 630–632, although having differentvolumetric dimensions, are designed to have substantially the samevolumetric size.

The ink reservoir sections 621, 633, in addition to cover 624 can bemade of plastic sections and are designed to be mated together by meansof heat sealing, ultra violet radiation, etc. Each of the equally sizedink reservoirs 630–632 is connected to a corresponding ink channel639–641 for allowing the flow of ink from the reservoir 630–632 to acorresponding ink output port 635–637. The ink reservoir 632 having inkchannel 641, and output port 637, the ink reservoir 631 having inkchannel 640 and output port 636, and the ink reservoir 630 having inkchannel 639 and output port 637.

In operation, the ink reservoirs 630–632 can be filled withcorresponding ink and the section 633 joined to the section 621. The inkreservoir sections 630–632, being collapsible bladders, allow for ink totraverse ink channels 639–641 and therefore be in fluid communicationwith the ink output ports 635–637. Further, if required, an air inletport can also be provided to allow the pressure associated with inkchannel reservoirs 630–632 to be maintained as required.

The cap 624 can be joined to the ink reservoir section 620 so as to forma pressurized cavity, accessible by the air pressure inlet port.

The ink reservoir sections 621, 633 and 624 are designed to be connectedtogether as an integral unit and to be inserted inside printer rollsections 622, 623. The printer roll sections 622, 623 are designed tomate together by means of a snap fit by means of male portions 645–647mating with corresponding female portions (not shown). Similarly, femaleportions 654–656 are designed to mate with corresponding male portions660–662. The paper roll sections 622, 623 are therefore designed to besnapped together. One end of the film within the role is pinched betweenthe two sections 622, 623 when they are joined together. The print filmcan then be rolled on the print roll sections 622, 625 as required.

As noted previously, the ink reservoir sections 620, 621, 633, 624 aredesigned to be inserted inside the paper roll sections 622, 623. Theprinter roll sections 622, 623 are able to be rotatable aroundstationery ink reservoir sections 621, 633 and 624 to dispense film ondemand.

The outer casing sections 626 and 627 are further designed to be coupledaround the print roller sections 622, 623. In addition to each end ofpinch rollers eg 612, 613 is designed to clip in to a correspondingcavity eg 670 in cover 626, 627 with roller 613 being driven externally(not shown) to feed the print film and out of the print roll.

Finally, a cavity 677 can be provided in the ink reservoir sections 620,621 for the insertion and gluing of an silicon chip integrated circuittype device 53 for the storage of information associated with the printroll 42.

As shown in FIG. 56, the print roll 42 is designed to be inserted intothe Artcam camera device so as to couple with a coupling unit 680 whichincludes connector pads 681 for providing a connection with the siliconchip 53. Further, the connector 680 includes end connectors of fourconnecting with ink supply ports 635–637. The ink supply ports are inturn to connect to ink supply lines eg 682 which are in turninterconnected to printheads supply ports eg. 687 for the flow of ink toprint-head 44 in accordance with requirements.

The “media” 611 utilised to form the roll can comprise many differentmaterials on which it is designed to print suitable images. For example,opaque rollable plastic material may be utilized, transparencies may beused by using transparent plastic sheets, metallic printing can takeplace via utilization of a metallic sheet film. Further, fabrics couldbe utilised within the printer roll 42 for printing images on fabric,although care must be taken that only fabrics having a suitablestiffness or suitable backing material are utilised.

When the print media is plastic, it can be coated with a layer whichfixes and absorbs the ink. Further, several types of print media may beused, for example, opaque white matte, opaque white gloss, transparentfilm, frosted transparent film, lenticular array film for stereoscopic3D prints, metallised film, film with the embossed optical variabledevices such as gratings or holograms, media which is pre-printed on thereverse side, and media which includes a magnetic recording layer. Whenutilising a metallic foil, the metallic foil can have a polymer base,coated with a thin (several micron) evaporated layer of aluminum orother metal and then coated with a clear protective layer adapted toreceive the ink via the ink printer mechanism.

In use the print roll 42 is obviously designed to be inserted inside acamera device so as to provide ink and paper for the printing of imageson demand. The ink output ports 635–637 meet with corresponding portswithin the camera device and the pinch rollers 672, 673 are operated toallow the supply of paper to the camera device under the control of thecamera device.

As illustrated in FIG. 56, a mounted silicon chip 53 is insert in oneend of the print roll 42. In FIG. 57 the authentication chip 53 is shownin more detail and includes four communications leads 680-683 forcommunicating details from the chip 53 to the corresponding camera towhich it is inserted.

Turning to FIG. 57, the chip can be separately created by means ofencasing a small integrated circuit 687 in epoxy and running bondingleads eg. 688 to the external communications leads 680–683. Theintegrated chip 687 being approximately 400 microns square with a 100micron scribe boundary. Subsequently, the chip can be glued to anappropriate surface of the cavity of the print roll 42. In FIG. 58,there is illustrated the integrated circuit 687 interconnected tobonding pads 681, 682 in an exploded view of the arrangement of FIG. 57.

Artcards can, of course, be used in many other environments. For exampleArtCards can be used in both embedded and personal computer (PC)applications, providing a user-friendly interface to large amounts ofdata or configuration information.

This leads to a large number of possible applications. For example, aArtCards reader can be attached to a PC. The applications for PCs aremany and varied. The simplest application is as a low cost read-onlydistribution medium. Since ArtCards are printed, they provide an audittrail if used for data distribution within a company.

Further, many times a PC is used as the basis for a closed system, yet anumber of configuration options may exist. Rather than rely on a complexoperating system interface for users, the simple insertion of a ArtCardsinto the ArtCards reader can provide all the configuration requirements.

While the back side of a ArtCards has the same visual appearanceregardless of the application (since it stores the data), the front of aArtCards is application dependent It must make sense to the user in thecontext of the application.

It can therefore be seen that the arrangement of FIG. 59 provides for anefficient distribution of information in the forms of books, newspapers,magazines, technical manuals, etc. In a further application, asillustrated in FIG. 60, the front side of a ArtCards 80 can show animage that includes an artistic effect to be applied to a sampled image.A camera system 81 can be provided which includes a cardreader 82 forreading the programmed data on the back of the card 80 and applying thealgorithmic data to a sampled image 83 so as to produce an output image84. The camera unit 81 including an on board inkjet printer andsufficient processing means for processing the sampled image data. Afurther application of the ArtCards concept, hereinafter called“BizCard” is to store company information on business cards. BizCard isa new concept in company information. The front side of a bizCard asillustrated in FIG. 61 and looks and functions exactly as today's normalbusiness card. It includes a photograph and contact information, with asmany varied card styles as there are business cards. However, the backof each bizCard contains a printed array of black and white dots thatholds 1–2 megabytes of data about the company. The result is similar tohaving the storage of a 3.5″ disk attached to each business card.

The information could be company information, specific product sheets,web-site pointers, e-mail addresses, a resume . . . in short, whateverthe bizCard holder wants it to. BizCards can be read by any ArtCardsreader such as an attached PC card reader, which can be connected to astandard PC by a USB port. BizCards can also be displayed as documentson specific embedded devices. In the case of a PC, a user simply insertsthe bizCard into their reader. The bizCard is then preferably navigatedjust like a web-site using a regular web browser.

Simply by containing the owner's photograph and digital signature aswell as a pointer to the company's public key, each bizCard can be usedto electronically verify that the person is in fact who they claim to beand does actually work for the specified company. In addition bypointing to the company's public key, a bizCard permits simpleinitiation of secure communications.

A further application, hereinafter known as “TourCard” is an applicationof the ArtCards which contains information for tourists and visitors toa city. When a tourcard is inserted into the ArtCards book reader,information can be in the form of:

-   -   Maps    -   Public Transport Timetables    -   Places of Interest    -   Local history    -   Events and Exhibitions    -   Restaurant locations    -   Shopping Centres

TourCard is a low cost alternative to tourist brochures, guide books andstreet directories. With a manufacturing cost of just one cent per card,tourCards could be distributed at tourist information centres, hotelsand tourist attractions, at a minimum cost, or free if sponsored byadvertising. The portability of the bookreader makes it the perfectsolution for tourists. TourCards can also be used at informationkiosk's, where a computer equipped with the ArtCards reader can decodethe information encoded into the tourCard on any web browser.

It is interactivity of the bookreader that makes the tourCard soversatile. For example, Hypertext links contained on the map can beselected to show historical narratives of the feature buildings. In thisway the tourist can embark on a guided tour of the city, with relevanttransportation routes and timetables available at any time. The tourCardeliminates the need for separate maps, guide books, timetables andrestaurant guides and creates a simple solution for the independenttraveller.

Of course, many other utilizations of the data cards are possible. Forexample, newspapers, study guides, pop group cards, baseball cards,timetables, music data files, product parts, advertising, TV guides,movie guides, trade show information, tear off cards in magazines,recipes, classified ads, medical information, programmes and software,horse racing form guides, electronic forms, annual reports, restaurant,hotel and vacation guides, translation programmes, golf courseinformation, news broadcast, comics, weather details etc.

For example, the ArtCards could include a book's contents or anewspaper's contents. An example of such a system is as illustrated inFIG. 59 wherein the ArtCards 70 includes a book title on one surfacewith the second surface having the encoded contents of the book printedthereon. The card 70 is inserted in the reader 72 which can include aflexible display 73 which allows for the folding up of card reader 72.The card reader 72 can include display controls 74 which allow forpaging forward and back and other controls of the card reader 72.

1. An image processing and printing apparatus which comprises aprocessor that is configured to process image data to apply effects tothe image data; image data input circuitry for receiving image data forprocessing; a card reader for reading a card having a two-dimensionalcode printed on one side of the card, configured to generate datarepresenting executable code upon reading the card, and connected to theprocessor to write the data to the processor, which is configured toexecute the code and to perform an image processing operation on theimage data according to the code; a printhead interface connected to theprocessor to receive processed image data from the processor and togenerate print data; and a pagewidth inkjet printhead connected to theprinthead interface to print processed images.
 2. An image processingand printing apparatus as claimed in claim 1, in which the image datainput circuitry includes a memory which is connected to an image capturedevice to receive image data from the image capture device, the memorybeing accessible by the processor to write the image data to theprocessor.
 3. An image processing and printing apparatus as claimed inclaim 2, in which the memory is a random access memory.
 4. An imageprocessing and printing apparatus as claimed in claim 3, which includesa reader motor for propelling the card past the linear sensor at asubstantially constant rate.
 5. An image processing and printingapparatus as claimed in claim 1, in which the card reader includes alinear sensor having a scanning resolution which is at least twice aresolution of the two-dimensional code on the card.
 6. An imageprocessing and printing apparatus as claimed in claim 1, in which theprocessor is configured to detect, via the card reader, a data area onthe card and subsequently to detect a bit pattern represented by thetwo-dimensional code in the data area.
 7. An image processing andprinting apparatus as claimed in claim 6, in which the processor isconfigured to perform a decoding operation on the bit pattern togenerate redundancy encoded data and then to decode the redundancyencoded data to generate the executable code.